Key Responsibilities

- Lead the design, development, and deployment of scalable data pipelines using AWS Glue, Apache Spark, and PySpark.

- Design and implement modern Lakehouse architecture leveraging Apache Iceberg.

- Build and maintain robust ETL/ELT pipelines using Python and dbt.

- Develop and manage workflow orchestration using Apache Airflow or AWS-native orchestration services.

- Design and optimize data models, ETL processes, and SQL queries for Amazon Redshift and analytical workloads.

- Establish best practices for CI/CD, source control using GitHub, and automated testing.

- Collaborate with business stakeholders, architects, and cross-functional teams to translate business requirements into technical solutions.

- Provide technical leadership, mentor engineering teams, and drive code quality, performance optimization, and architectural best practices.

- Ensure data quality, governance, reliability, and operational excellence across data platforms.





Skills Required

Skills and Experience

- 8+ years of experience in Data Engineering, with experience leading technical teams.

- Strong hands-on expertise in AWS Glue, Apache Spark, PySpark, and Python.

- Experience with dbt for data transformation, testing, and documentation.

- Hands-on knowledge of Apache Iceberg and Lakehouse architecture.

- Experience with Apache Airflow for workflow orchestration.

- Solid expertise in Amazon Redshift, SQL optimization, and performance tuning.

- Experience with GitHub, GitHub Actions, Jenkins, AWS CodePipeline, or similar CI/CD tools.

- Solid understanding of data modeling, partitioning strategies, and performance optimization.

- Excellent problem-solving, communication, and stakeholder management skills.

📌 Lead Data Engineer (Bengaluru)
🏢 Flexing It®
📍 Bengaluru