Job Summary

ETL Data - Senior Engineer

Location: Noida, UP, India

Responsibilities

- Design scalable data engineering solutions using PySpark and modern distributed data processing frameworks.
- Define data ingestion, transformation, and processing architectures aligned with business and analytical objectives.
- Design and optimize Snowflake or Delta Lake on Databricks solutions to support enterprise-scale data platforms.
- Lead implementation of high-performance batch and streaming data pipelines.
- Define data streaming standards, integration frameworks, and scalable processing patterns.
- Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows.
- Establish monitoring, scheduling, and operational controls for reliable pipeline execution.
- Drive data quality, validation, reconciliation, and governance practices across data engineering solutions.
- Design data engineering solutions following up-to-date Lakehouse architecture principles, data observability practices, and platform engineering standards to improve scalability, reliability, and operational visibility.
- Drive development of business-focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption across analytical platforms.
- Promote responsible use of AI-assisted engineering capabilities to improve development productivity, testing, documentation, and engineering quality.




- Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards.
- Troubleshoot complex data processing, workflow, and streaming platform issues through detailed root cause analysis.
- Collaborate with various teams and stakeholders to support end-to-end data platform delivery.

Mandatory Competencies

- Big Data - Big Data - PySpark
- Data Science and Machine Learning - Data Science and Machine Learning - Apache Spark
- Data AI - Data Engineering - Data Quality Validation
- Data Science and Machine Learning - Data Science and Machine Learning - Python
- Database - Database Programming - SQL
- Cloud - AWS - AWS SNS, AWS SQS, AWS Kinesis
- Data Science and Machine Learning - Data Science and Machine Learning - Databricks
- Cloud - AWS - Amazon CloudWatch
- Cloud - AWS - AWS S3, S3 glacier, AWS EBS
- Cloud - AWS - Amazon IAM, AWS Secrets Manager, AWS KMS, AWS Cognito
- Cloud - AWS - Amazon API Gateway
- Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS Redshift
- Cloud - AWS - Dynamo DB, Amazon Aurora, Amazon RDS
- Cloud - AWS - Amazon EC2 / Autoscalaing / Load Balancing, AWS App Runner
- Data Engineering - ETL / Data Integration - ETL Data Integration
- Middleware - API Middleware - API (SOAP, REST)
- Beh - Communication and collaboration

📌 ETL Data - Senior Engineer (Noida)
🏢 IRIS SOFTWARE
📍 Noida