Job Description
Job Title: Data Engineer - Databricks (AWS) | PySpark | Delta Lake
nLocation: Bengaluru, Noida, Gandhi Nagar, Pune, Chennai
nExperience: 4+ Years
n

nJob Summary
nWe are looking for a talented Data Engineer with strong expertise in Databricks on AWS, PySpark, Delta Lake, and up-to-date data engineering practices. The ideal candidate will be responsible for building scalable data pipelines, implementing data governance frameworks, optimizing platform performance, and supporting enterprise analytics initiatives within the Healthcare/Life Sciences domain.
n

nKey Responsibilities
n
- n
- Design, develop, and maintain scalable data pipelines using Databricks on AWS.n
- Build and support batch and streaming data ingestion frameworks.n
- Implement data quality, governance, lineage, and security controls using Unity Catalog and Databricks best practices.n
- Develop and optimize ETL/ELT processes using PySpark and SQL.n
- Design and implement Medallion Architecture (Bronze, Silver, Gold) data solutions.n




- Build and maintain CDC-based ingestion pipelines using AWS and Databricks services.n
- Optimize Spark performance, cluster utilization, and platform costs.n
- Implement and support Delta Lake features to ensure data reliability and governance.n
- Manage Structured Streaming, Auto Loader, and real-time data processing requirements.n
- Develop CI/CD processes using Git, Databricks Asset Bundles, and deployment automation tools.n
- Monitor production environments, troubleshoot issues, and ensure SLA compliance.n
- Collaborate with business stakeholders, analysts, and engineering teams to deliver high-quality data solutions.n
- Mentor junior engineers and contribute to data engineering best practices.n

n

nRequired Skills & Qualifications
nTechnical Skills
n
- n
- 4+ years of experience in Data Engineering.n
- 4+ years of hands-on experience with Databricks on AWS.n
- Solid expertise in PySpark and Spark optimization techniques.n
-

📌 AWS Data Engineer (Lapkaman)
🏢 Tata Consultancy Services
📍 Lapkaman