We are seeking an experienced Data Engineer with strong expertise in AWS, Databricks, Python, PySpark, and SQL to build and optimize scalable data platforms and up-to-date data pipelines. The ideal candidate will have hands-on experience in cloud-native data engineering, ETL/ELT development, and large-scale data processing across AWS environments.

Key Responsibilities
- Design, develop, and maintain scalable data ingestion, transformation, and integration pipelines.
- Build and optimize ETL/ELT workflows using Python, PySpark, SQL, and Databricks.
- Develop cloud-native data solutions on AWS platforms.
- Create and maintain data pipelines for structured and unstructured data processing.
- Work with AWS services including Glue, Redshift, S3, Lambda, EMR, and Databricks.
- Collaborate with architects, analysts, and business stakeholders to design data models and integration frameworks.
- Implement data quality checks, monitoring, and governance controls.
- Develop reusable frameworks and components for orchestration, automation, and monitoring.




- Optimize Spark workloads and data processing performance.
- Troubleshoot data pipeline issues and ensure reliability, scalability, and security.
- Support Agile delivery and contribute to continuous improvement initiatives.

Required Skills
- 6-10 years of experience in Data Engineering.
- Strong expertise in Python programming.
- Hands-on experience with PySpark and Apache Spark.
- Strong SQL development and optimization skills.
- Experience with Databricks platform.
- Experience building ETL/ELT pipelines.
- Strong AWS experience including:
- AWS Glue
- Amazon Redshift
- Amazon S3
- AWS Lambda
- Amazon EMR
- Knowledge of data modelling and data integration concepts.
- Experience working with large-scale data platforms and cloud-native architectures.

Preferred Skills
- Data Lake and Data Warehouse implementations.
- Workflow orchestration tools.
- Real-time data processing experience.
- Data Governance and Data Quality framewo

📌 Data Engineer (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru