Role: Databricks Engineer
Experience: 9-12 Years
Location: ALL EXL Locations
Work Mode: Hybrid
Key Role and Responsibilities:
 Design, build, and maintain Databricks workspaces, clusters, and compute pools across development, testing, and production environments.
 Configure and manage Unity Catalog for data governance, fine-grained access control, permissions, metadata management, and data lineage.
 Optimize Databricks cluster configurations, including instance types, auto-scaling, spot/preemptible nodes, and compute pools to improve performance and reduce costs.
 Implement workspace best practices, including folder structures, access controls, secret management using Databricks Secrets, Azure Key Vault, or AWS Secrets Manager.
 Create, schedule, and manage Databricks Jobs, Workflows, and multi-task job orchestration with dependency management.
 Design and implement Delta Lake tables using partitioning, Z-Ordering, OPTIMIZE, VACUUM, and file compaction techniques.
 Build and maintain Medallion Architecture (Bronze, Silver, and Gold layers)



for scalable and governed data lakehouse solutions.
 Develop Delta Live Tables (DLT) pipelines with built-in data quality expectations for reliable ETL/ELT processing.
 Manage schema evolution, table versioning, Time Travel, and Change Data Feed (CDF) to support incremental data processing.
 Design and implement lakehouse architectures integrating Delta Lake with cloud storage and external systems such as Azure Data Lake Storage (ADLS), Kafka, Event Hubs, and Kinesis.
 Develop scalable batch and real-time data pipelines using PySpark, Spark SQL, Structured Streaming, and Delta Lake.
 Build streaming ingestion pipelines from Kafka, Azure Event Hubs, and other streaming platforms into Delta tables.
 Optimize PySpark applications using broadcast joins, Adaptive Query Execution (AQE), energetic partition pruning, caching, and Photon Engine.
 Develop reusable transformation frameworks, utility libraries, and pipeline templa

📌 Senior Manager (Pune)
🏢 EXL
📍 Pune