Azure Data Engineer

Role Overview

An Azure Data Engineer will design, build, and support enterprise-scale cloud data platforms. This role requires hands-on expertise in Azure, Databricks, Spark, Lakehouse architecture, and modern data pipelines, along with the ability to collaborate with engineers, business stakeholders, and DevOps teams to deliver secure, reliable, analytics-ready data solutions in an Agile workplace.

Key Responsibilities

- Architect and deliver scalable data lakehouse solutions using Azure Data Lake Storage, Azure Databricks, Delta Lake, Unity Catalog, and Medallion architecture.
- Build curated data products, data marts, and analytics-ready datasets for analysts, data scientists, and enterprise reporting teams.
- Develop reliable batch and near-real-time pipelines using Databricks, Apache Spark, PySpark, SQL, Azure Data Factory, Synapse, and related Azure services.
- Implement ingestion frameworks for structured, semi-structured, and unstructured data while ensuring data quality, lineage, validation, observability, and reusable patterns.
- Apply data modeling, partitioning, schema evolution, performance tuning, cost optimization, error handling, and operational readiness practices.
- Implement CI/CD for data workloads using Azure DevOps or GitHub Actions, including automated testing, deployment, release governance, and secure engineering practices.
- Use Entra ID, RBAC, secrets management, Key Vault, Managed Identity, and governance controls to protect data platforms.
- Leverage GitHub Copilot, Microsoft Copilot, Databricks Assistant,



and AI-powered workflows to accelerate SQL/PySpark development, ETL delivery, test data generation, documentation, troubleshooting, and productivity.

Candidate Qualifications

- Expert-level experience with Azure Databricks, Azure Synapse Analytics, Azure Data Factory, PySpark, Spark SQL, and Azure Data Lake Storage Gen2.
- Strong knowledge of Delta Lake, Lakehouse architecture, Unity Catalog, data governance, Medallion architecture, CDC, SCD, data quality frameworks, reconciliation, and lineage.
- Advanced SQL expertise across Oracle, SQL Server, PostgreSQL, and modern cloud data platforms.
- Hands-on experience with Informatica PowerCenter is preferred, especially for migration programs.
- Experience with Git, GitHub, Azure DevOps, CI/CD, automated deployment, secure coding, monitoring, alerting, and operational dashboards.
- Strong understanding of Azure security services including Key Vault, Managed Identity, Entra ID, and role-based access controls.

Tech Stack

- Data Engineering: Azure Databricks, Azure Data Factory, Azure Synapse, ADLS Gen2, Python, PySpark, SQL, Spark SQL, ETL/ELT, orchestration, metadata management, and data modeling.
- Lakehouse: Delta Lake, Bronze/Silver/Gold layers, schema evolution, SCD, reusable ingestion frameworks, and performance tuning.
- DevOps & Quality: Git, GitHub, Azure DevOps, CI/CD, automated testing, release pipelines, SonarQube, secure coding, Azure Monitor, Application Insights, logs, alerts, and pipeline health checks.

📌 Azure Data Engineer (Hyderabad)
🏢 MetLife
📍 Hyderabad