Mandatory Skills:

Python, Apache Spark, Data Quality & Validation, PySpark, SQL, Amazon Kinesis, Apache Kafka, Databricks Workflows, Delta Lake on Databricks

Additional Skills:

AWS S3 + Amazon Aurora + DynamoDB, AWS Glue, AWS Redshift, Data Governance Frameworks, GitHub

Key Responsibilities
Design scalable data engineering solutions using PySpark and contemporary distributed data processing frameworks.
Define data ingestion, transformation, and processing architectures aligned with business and analytical objectives.

Behavioral Competencies
Demonstrates solid ownership while driving data engineering excellence.
Collaborate effectively with various teams and business stakeholders to ensure smooth delivery.

Must Have
Python & Advanced SQL
• Spark / distributed processing / PySpark
• Cloud platforms (AWS) Modules like Lambda, Glue, EMR, Dynamo DB, S3, Redshift, EC2, RDS, Step Functions, Cloud Watch, Systems Manager, Event Bridge, DMS
• SSIS and experience with C# scripting
• Data pipeline automation

Nice to have
• Kafka/Kinesis streaming
• Basic knowledge of AI
• Power BI
Postgre SQL
DBT

📌 Data Engineer Lead Noida
🏢 IRIS SOFTWARE
📍 Noida