Role & Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python, PySpark, SQL, AWS Glue, Azure Data Factory, and Apache Airflow.
- Build and manage cloud-based data platforms across AWS and Microsoft Azure, supporting enterprise data engineering and analytics requirements.
- Develop data ingestion pipelines to collect data from databases, APIs, files, streaming platforms, and other structured and semi-structured sources.
- Implement data processing and transformation solutions using PySpark, Apache Spark, Python, and SQL.
- Develop and optimize data lakes and data warehouses using Amazon S3, Amazon Redshift, Snowflake, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
- Work with AWS services including AWS Glue, Lambda, EMR, Athena, S3, Redshift, Step Functions, SQS, SNS, and CloudWatch.
- Work with Azure services including Azure Data Factory, Azure Databricks, ADLS Gen2, Azure Synapse Analytics, Azure Functions, and Azure Monitor.
- Design and implement scalable data models, including fact tables, dimension tables, star schemas,



and analytical data models.
- Develop complex SQL queries, stored procedures, transformations, and data integration processes for reporting and analytics.
- Implement data quality, validation, reconciliation, and monitoring frameworks to ensure data accuracy, completeness, and reliability.
- Optimize Spark jobs, SQL queries, ETL workflows, and cloud resources for improved performance and cost efficiency.
- Develop Python-based automation scripts, data processing frameworks, utilities, and integrations.
- Configure and maintain workflow orchestration using Apache Airflow for reliable scheduling, dependency management, and pipeline monitoring.
- Collaborate with business analysts, data architects, developers, and stakeholders to understand requirements and deliver scalable data solutions.
- Support analytics, reporting, and BI initiatives by providing trusted, curated, and high-quality datasets.

📌 Data Engineer (Bengaluru)
🏢 Aetherrise Solutions
📍 Bengaluru