Role & Responsibilities
Design, develop, and maintain scalable ETL/ELT data pipelines using Python, PySpark, SQL, AWS Glue, Azure Data Factory, and Apache Airflow.
Build and manage cloud-based data platforms across AWS and Microsoft Azure, supporting enterprise data engineering and analytics requirements.
Develop data ingestion pipelines to collect data from databases, APIs, files, streaming platforms, and other structured and semi-structured sources.
Implement data processing and transformation solutions using PySpark, Apache Spark, Python, and SQL.
Develop and optimize data lakes and data warehouses using Amazon S3, Amazon Redshift, Snowflake, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
Work with AWS services including AWS Glue, Lambda, EMR, Athena, S3, Redshift, Step Functions, SQS, SNS, and CloudWatch.
Work with Azure services including Azure Data Factory, Azure Databricks, ADLS Gen2, Azure Synapse Analytics, Azure Functions, and Azure Monitor.
Design and implement scalable data models, including fact tables, dimension tables, star schemas,



and analytical data models.
Develop complex SQL queries, stored procedures, transformations, and data integration processes for reporting and analytics.
Implement data quality, validation, reconciliation, and monitoring frameworks to ensure data accuracy, completeness, and reliability.
Optimize Spark jobs, SQL queries, ETL workflows, and cloud resources for improved performance and cost efficiency.
Develop Python-based automation scripts, data processing frameworks, utilities, and integrations.
Configure and maintain workflow orchestration using Apache Airflow for reliable scheduling, dependency management, and pipeline monitoring.
Collaborate with business analysts, data architects, developers, and stakeholders to understand requirements and deliver scalable data solutions.
Support analytics, reporting, and BI initiatives by providing trusted, curated, and high-quality datasets.

📌 Data Engineer Bengaluru
🏢 Aetherrise Solutions
📍 Bengaluru