: Lead Data Engineer (Level 4)
Location – Chennai
Experience - 5 Years
Role Summary
We are seeking an experienced Lead Data Engineer (Level 4) to serve as the technical leader responsible for the modernization and future-state architecture of our enterprise Sales Validation platform. In this role, you will lead the end-to-end technical delivery, drive a critical platform migration from Azure Synapse to Databricks, establish engineering excellence standards, and mentor high-performing data engineering teams.
Key Responsibilities
● Architecture & Modernization: Lead Databricks architecture, future-state design, and delivery across major platform workstreams.
● Platform Migration: Direct and execute the end-to-end migration of legacy ETL/ELT workloads from Azure Synapse to Databricks.
● Engineering Standards & Frameworks: Establish robust engineering standards, design reusable frameworks, and implement best practices for Lakehouse architecture.
● Technical Ownership & Optimization: Own technical delivery across key workstreams, driving performance tuning, scalability, and system reliability.
● Leadership & Quality Assurance: Conduct thorough architecture/design reviews, enforce code quality via code reviews, and mentor junior to mid-level engineers.
Required Qualifications & Technical Skills
- Databricks & Lakehouse (Core Focus)
● Deep expertise in Databricks and Lakehouse Architecture.
● Advanced hands-on experience with PySpark, Spark SQL, and Delta Lake.
● Production experience implementing Unity Catalog for governance and access management.
● Solid knowledge of orchestration and automation using Databricks Workflows and Databricks Asset Bundles (DABs).
2. Azure Data Platform
● Proven experience building and orchestrating pipelines using Azure Data Factory (ADF).
● Strong knowledge of Azure Data Lake Storage Gen2 (ADLS Gen2), Azure SQL, and enterprise Azure Security Patterns.
3. Migration & Legacy Modernization
● Hands-on experience modernizing legacy ETL workloads and migrating from Azure Synapse Analytics to Databricks.
● Deep practical knowledge of Synapse components: Synapse Pipelines, Synapse Spark, Synapse SQL Pools, Synapse Notebooks, and Synapse Data Flows.
4. Core Software & Data Engineering
● Advanced SQL and complex Data Modeling capabilities.
● Proven track record in query and job performance tuning and optimization.
● Experience with CI/CD automation and version control using GitLab CI/CD.
📌 Lead Data Engineer (Chennai)
🏢 Recro
📍 Chennai