Dear Professionals

Greetings from Tata consultancy Services,

Job Title : AWS Data Engineer With Spark and Scala

Experience : 6-10 years

Location: Hyderabad/Chennai/ Bangalore/Pune/Delhi

Mode of Work : Work from Office

Mode of Interview: Virtual

Key Responsibilities
- Pipeline Development: Design, build, and maintain highly scalable batch and real-time streaming data pipelines using Apache Spark and Scala.
- AWS Data Solutions: Architect and deploy modern data platform components leveraging AWS services including S3, EMR, Glue, Redshift, Athena, and Lambda.
- Performance Tuning: Optimize and tune Spark application engines for high-volume datasets consisting of billions of records.
- Orchestration & Workflow: Build, manage, and monitor workflow dependencies using orchestration tools like Apache Airflow or AWS Step Functions.
- Data Modeling & Quality: Implement efficient data schemas, Medallion Architecture (Bronze/Silver/Gold layers), and automated data quality checks.
- Troubleshooting & Ops:



Perform root cause analysis on production pipeline failures and debug complex distributed computing bottlenecks.
- Governance & Compliance: Enforce enterprise-grade data security, IAM policy frameworks, and data governance standards.
- Technical Leadership: Collaborate with architects, manage stakeholder expectations, and provide technical guidance to a small squad of data engineers.

Required Technical Skills
- Core Experience: 6+ years in dedicated Data Engineering or Big Data roles.
- Programming: Absolute proficiency in Scala (Mandatory) along with a robust foundation in SQL.
- Big Data Frameworks: In-depth knowledge of Spark Core, Spark SQL, DataFrames/Datasets, and Hadoop ecosystems (HDFS, Hive, YARN).
- AWS Cloud Services: Extensive experience configuring and optimizing managed big data workloads on AWS EMR, Glue, and Redshift.
- Databases & Storage: Hands-on experience dealing with massive structured/unstructured datasets across SQL, NoSQL (e.g., MongoDB, Cassand

📌 Aws Data Engineer (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru