Skills Required: Digital: Microsoft Azure~Digital : Scala~Digital : Databricks
Experience: 8-10 Years
Data Engineer Databricks & Azure Job Summary
- We are seeking an experienced Data Engineer with strong expertise in Azure Data Services and Databricks to design| build| and maintain scalable| high performance data pipelines.
- The ideal candidate will work closely with analytics| data science| and business teams to enable reliable data ingestion| transformation| and analytics in a cloud native environment.
- Key Responsibilities- Data Engineering & ETL Design| develop| and maintain end to end data pipelines using Azure Databricks Build scalable ETL/ELT solutions for structured and unstructured data Process large datasets using Spark (PySpark / Scala / SQL)
- Optimize Databricks jobs for performance and cost efficiency Implement data transformations using Delta Lake Azure Data Platform Work with Azure services such as: Azure Data Lake Storage (ADLS Gen2)o Azure Data Factory (ADF)o Azure Databrickso Azure Synapse Analyticso Azure SQL / SQL Server
- Design data architectures following Azure best practices Implement secure and scalable cloud native data solutions
- Data Quality| Governance & Reliability Ensure data accuracy| consistency| and reliability Implement data validation| monitoring| and error handling Support data governance| lineage| and audit requirements Apply security best practices and access controls
- Performance & Optimization Tune Spark jobs and queries for performance improvements Optimize partitioning| caching| and cluster configurations Monitor pipeline health and troubleshoot failures
- Collaboration & Process Work closely with data analysts| data scientists| and business teams Participate in Agile/Scrum ceremonies Document data pipelines| architectures| and best practices Support production deployments and ongoing enhancements
- Required Skills &
Qualifications Technical Skills Strong experience with Azure Databricks Proficiency in Spark SQL / PySpark / Scala Hands on experience with Azure Data Factory
- Robust knowledge of Azure Data Lake (ADLS Gen2) Experience with Delta Lake Solid understanding of SQL Familiarity with ETL design patterns Experience with CI/CD pipelines (Azure DevOps) Knowledge of Git
Rate Details
IND_Developer
Category 4
Level III
Rs90,000.00 - Rs165,000.00 Monthly NOOT
📌 Digital : Microsoft Azure~Digital : Scala~Digital : Databricks (India)
🏢 Clifyx
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.