JD – Snr Software Engineer (ETL)

Experience & Expectations :
- Leverage extensive experience (4 to 8 years overall ETL experience, to assist in solution design and delivery along with build of new ETLs).
- We are seeking an experienced ETL Developer with strong expertise in Big Data (Spark, Cloudera).
- Experience orchestrating workflows using AWS Step Functions (state machines) for reliable and scalable data pipelines.
- Ability to implement end-to-end serverless data architectures integrating Glue, Lambda, S3, and Redshift

Core Responsibilities :
- Build and maintain high volume ETL/ELT pipelines across Hadoop (HDFS, Hive, Spark, Kafka) and AWS (Glue, EMR, Lambda, Step Functions, Redshift).
- Develop distributed data processing solutions using PySpark, Spark SQL, and scalable cloud serverless patterns.
- Implement reusable data ingestion frameworks for batch, ability to design & implement Orchestration process and Leverage AI
- Optimize data workflows using partitioning, bucketing, compression, file formats (Parquet/ORC).




- Understanding hybrid data lake architectures using S3 + HDFS, ensuring data governance and best practices are adheres
- Experience to deliver complex projects in an Agile environment
- Assist in Design and build the robust, scalable and secure software solutions across the having no/least adoption
- Define transparent technical specifications and make architecture decisions that align with business goals and long-term scalability.
- Implement best practices (including secure code guidelines) through the implementation of unit tests, automation, leverage and code reviews. Drive continuous improvement in code quality and maintainability.
- Troubleshooting issues and proactively solving problems as they arise, ensuring the smooth operation of full stack applications
- Ability to understand the data flow diagram, data modelling and Lineages
- Job orchestration using Airflow, Control M, Step Functions, or event-driven triggers.
- Ensure data is protecte

📌 Snr Data Engineer (Noida)
🏢 Alight Solutions
📍 Noida