Key Responsibilities:

- Design, develop, and maintain scalable data pipelines on AWS.
- Build and optimize ETL/ELT workflows using PySpark, AWS Glue, and EMR.
- Develop data ingestion frameworks for structured and unstructured data.
- Design and implement Data Lake and Data Warehouse solutions.
- Process large-scale datasets using Spark and distributed computing techniques.
- Create and maintain data models supporting reporting and analytics.
- Ensure data quality, governance, security, and compliance standards.
- Monitor, troubleshoot, and optimize production data pipelines.
- Implement CI/CD practices for data engineering solutions.
- Collaborate with cross-functional teams in Agile environments.

Mandatory Technical Skills:

1) AWS Services: Amazon S3, AWS Glue, EMR, Lambda, Athena, Redshift, CloudWatch, IAM, Step Functions

2) Data Engineering: Data Lakes,



Data Warehousing, ETL / ELT Development, Batch Processing, Data Modeling, Data Migration

3) Big Data Technologies: Apache Spark, PySpark, Spark SQL

4) Programming: Python, SQL, Shell Scripting

5) DevOps & Automation: Git, CI/CD Pipelines, Infrastructure as Code (Terraform preferred)

Valuable-to-Have Skills:

- Databricks
- Snowflake
- Kafka
- Airflow
- Hadoop Ecosystem
- Docker
- Kubernetes
- Delta Lake
- Streaming Data Pipelines
- BFSI Domain Experience

Desired Technical Competencies:

- Distributed Data Processing
- Performance Tuning
- Cloud Architecture
- Data Security & Governance
- Data Integration Patterns

📌 Aws Data Engineer (6 years To 12 years) (Mumbai)
🏢 Tata Consultancy Services
📍 Mumbai