Senior Data Engineer – Databricks
? Location: Gurugram / Bangalore
? Work Mode: Work from Office (5 Days a Week)
Role Overview
We are seeking an experienced Senior Data Engineer with robust expertise in Databricks, PySpark, and modern Lakehouse architectures. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data platforms that support analytics, reporting, and machine learning workloads. This role requires deep hands-on experience with distributed data processing, scalable data engineering practices, and cloud-based data ecosystems.
Key Responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Spark SQL, and Delta Lake.
Build and optimize data ingestion frameworks, transformation logic, and end-to-end data workflows for both batch and real-time streaming use cases.
Implement and manage Delta Lake architectures, including schema evolution, data versioning, and ACID-compliant processing.
Collaborate with business stakeholders, data analysts,
and data scientists to gather requirements and deliver scalable data solutions.
Optimize Databricks clusters, notebooks, workflows, and jobs to improve performance, reliability, and cost efficiency.
Integrate Databricks with cloud-native services such as Azure Data Lake Storage (ADLS), AWS S3, Event Hubs/Kafka, Synapse, Redshift, and other enterprise data platforms.
Develop data quality, validation, monitoring, and observability frameworks to ensure data reliability and governance.
Contribute to data modeling, metadata management, and best practices for enterprise data platforms.
Support advanced analytics, AI/ML, and reporting initiatives by delivering high-quality and trusted datasets.
Participate in code reviews, CI/CD implementation, and platform optimization initiatives.
Required Skills & Experience
Must Have
8+ years of overall experience in Data Engineering, with significant hands-on expertise in Databricks.
Strong
📌 Senior Data Engineer (Gurugram)
🏢 EXL
📍 Gurugram