We are seeking an experienced Senior Data Engineer to design, build, and support scalable data integration and processing solutions on Azure and Databricks. Candidate should have solid hands-on experience developing enterprise ETL frameworks, implementing complex data transformations in Databricks, handling metadata-driven processing, and delivering reliable data pipelines using Azure technologies.

Candidate will be responsible for designing and implementing end-to-end data workflows, from file ingestion and data validation through business-facing SharePoint integrations and downstream system exports.

Required Technical Skills

Must Have:

Databricks (Unity Catalog)

PySpark and Spark SQL

Azure Data Factory (ADF) with parameterized orchestration

Delta Lake and Delta Tables

Metadata-driven ETL framework development

Data validation, quality, and reconciliation processes

Complex data transformation and data modeling

Git-based source control and CI/CD practices

Performance tuning and optimization of Databricks workloads

Data governance using Unity Catalog

Good to Have:

SharePoint Lists and SharePoint integration development

Power Automate

Azure monitoring and observability tools

📌 Senior Data Engineer | Azure + Databricks (Bengaluru)
🏢 Innova ESI
📍 Bengaluru