Azure Databricks Data Engineer
Location: 400013, Mumbai, Maharashtra, India
Experience: 5+ Years
Contact / CV Submission:
[email protected]
About the Role
We are looking for a skilled Data Engineer with strong hands‑on experience in Azure Data Platform , including Azure Databricks , Azure Data Factory (ADF) , Azure Synapse , and modern data engineering practices. You will design, build, and optimize scalable data pipelines and collaborate closely with architects, analysts, and business teams to enable data-driven decision-making.
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines using Azure Databricks , ADF , and other Azure data services.
- Build scalable ETL/ELT workflows for ingesting, transforming, and loading data from structured and unstructured sources.
- Implement Delta Lake architectures, medallion designs (Bronze/Silver/Gold layers), and best practices in Databricks.
- Optimize big data processing using Spark (PySpark/Scala).
- Manage data orchestration, scheduling, and automation using ADF and Databricks Jobs.
- Work with Azure components such as Azure Data Lake Storage (ADLS) , Azure SQL , Key Vault , and Azure Event Hub or Service Bus .
- Ensure data quality, reliability, lineage, and governance across the data lifecycle.
- Build CI/CD pipelines for data workloads using Azure DevOps or Git-based workflows.
- Collaborate with stakeholders to translate business needs into scalable technical solutions.
- Monitor and improve performance of data pipelines and cloud resources.
- Ensure compliance with security, privacy, and organizational data standards.
Required Skills & Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or related field.
- 5+ years of experience as a Data Engineer or similar role.
- Robust hands‑on experience with Azure Databricks (PySpark/Scala/Spark SQL).
- Proven experience developing pipelines using Azure Data Factory .
- Expertise in Azure Data Lake , Delta Lake , and modern data lakehouse architecture.
- Solid understanding of Spark performance tuning, partitioning, caching, and optimization strategies.
- Experience with Azure DevOps , Git repos, CI/CD for data workflows.
- Proficiency in SQL , including performance tuning and complex query building.
- Knowledge of data modeling , ETL/ELT frameworks , and data warehousing concepts .
- Experience with monitoring , alerting , and cost optimization in cloud environments.
Preferred / Nice-to-Have
- Experience with Azure Synapse Analytics (SQL Pools, Pipelines).
- Familiarity with Databricks Unity Catalog and advanced security controls.
- Exposure to streaming technologies (Structured Streaming, Event Hub, Kafka).
- Knowledge of Python for data engineering automation scripts.
- Understanding of MLflow , feature stores, or machine learning pipeline integration.
📌 Azure Databricks Data Engineer (Mumbai)
🏢 EliteSquad.AI
📍 Mumbai