Location: Chennai
Client: US-based IT services and consulting firm
Work Mode: Work from Office (5days/week)
Experience: 10+ Years
Employment Type: Full time
About the Role
We are looking for an experienced Senior Data Engineer with deep expertise in Databricks, Python, Apache Spark, SQL, and Azure to design and deliver enterprise-scale Lakehouse solutions. This role requires hands-on experience in building scalable data platforms, modern ETL/ELT pipelines, and cloud-native data engineering solutions.
The ideal candidate will have strong architectural experience, excellent stakeholder management skills, and a passion for building high-performance data platforms that support analytics and AI initiatives.
Key Responsibilities
- Design and implement enterprise-grade Lakehouse solutions using Databricks.
- Build scalable batch and real-time data pipelines using Apache Spark (PySpark) and SQL.
- Develop ETL/ELT pipelines, metadata-driven ingestion frameworks, and incremental data processing.
- Implement and optimize Databricks features including Delta Lake, Delta Live Tables, Autoloader, Structured Streaming, Unity Catalog, and Workflows.
- Design and maintain enterprise data warehouse solutions using 3NF and dimensional data modeling.
- Establish data governance, security, data quality, and metadata management frameworks.
- Lead ML lifecycle management using MLflow and support AI initiatives such as RAG and AI/BI.
- Deploy and manage cloud-native data solutions on Microsoft Azure.
- Build CI/CD pipelines and automate deployments using Azure DevOps.
- Optimize Spark workloads for scalability, reliability, and performance.
- Integrate enterprise applications such as ServiceNow with data platforms.
- Collaborate with business stakeholders to translate requirements into scalable technical solutions.
- Mentor engineering teams and provide technical leadership throughout project delivery.
Required Skills
- 10+ years of so
📌 Senior Data Engineer – Databricks, Python (Chennai)
🏢 Recruiterv
📍 Chennai
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.