Azure Data Engineer Azure Databricks, ADF & Azure Data Platform
The Team
Deloitte's Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.
Your Work Profile
The Azure Data Engineer is responsible for designing, developing, and maintaining scalable data platforms and data pipelines on Microsoft Azure. This role will work closely with business stakeholders, data architects, analysts, and application teams to deliver reliable, secure, and high-performing data solutions.
The ideal candidate should have strong hands-on experience with Azure Databricks, Azure Data Factory (ADF), and modern cloud-based data engineering practices.
Data Engineering & Development
Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Data Factory and Azure Databricks.
Build and optimize ETL/ELT processes for structured, semi-structured, and unstructured data.
Develop reusable frameworks and accelerators to improve data engineering productivity.
Implement data quality checks, reconciliation processes, and monitoring mechanisms.
Develop and maintain data models supporting reporting, analytics, and AI/ML workloads.
Ensure adherence to coding standards, security policies, and governance frameworks.
Data Platform & Cloud Engineering
Design and implement Lakehouse architectures using Azure Databricks and Azure Data Lake Storage Gen2.
Integrate data from various enterprise applications,
databases, APIs, and external sources.
Optimize Spark workloads and Databricks jobs for performance and cost efficiency.
Support CI/CD implementation using Azure Dev Ops.
Work with solution architects to design scalable and future-ready data platforms.
Participate in production support, troubleshooting, and root cause analysis activities.
Stakeholder & Project Collaboration
Collaborate with business users to understand data requirements and translate them into technical solutions.
Work closely with architects, analysts, and reporting teams to ensure data availability and accuracy.
Participate in Agile ceremonies including sprint planning, daily stand-ups, and retrospectives.
Prepare technical documentation, data flow diagrams, and operational runbooks.
Support testing, deployment, and release management activities.
Key Skills Required
Strong hands-on experience in Azure Databricks development using PySpark and SQL.
Experience building enterprise data pipelines using Azure Data Factory (ADF).
Strong understanding of Data Lake, Lakehouse, and Data Warehouse concepts.
Experience working with large-scale data processing and optimization techniques.
Proficiency in SQL and data modelling concepts.
Experience with CI/CD implementation using Azure Dev Ops.
Experience with Azure Data Lake Storage Gen2 (ADLS Gen2).
Strong understanding of Delta Lake architecture and optimization techniques.
Experience in developing and troubleshooting ETL/ELT pipelines.
Knowledge of data governance, security, and access management concepts.
Robust analytical, troubleshooting, and problem-solving skills.
Excellent communication and stakeholder management skills.
Key Skills & Competencies
Azure Databricks
PySpark
SQL
Azure Data Factory (ADF)
Azure Data Lake Storage Gen2 (ADLS)
Delta Lake
Data Warehousing Concepts
Data Modelling
ETL/ELT Development
Performance Optimization
Azure Dev Ops
Git Version Control
Data Quality & Validation
CI/CD Implementation
Agile/Scrum Methodology
📌 Azure Data Engineer (India)
🏢 Arminus
📍 India