Updated Job Description – Senior Azure Data Engineer

Location: Hyderabad

Summary

Seeking a Senior Data Engineer to architect and deliver enterprise-scale Azure data platforms, lead modernization and migration initiatives, and mentor engineering teams. The ideal candidate should possess deep expertise in Azure Databricks, Azure Synapse, Data Lakehouse architecture, PySpark, and contemporary data engineering practices. In addition, the candidate should effectively leverage AI-powered engineering tools and agents to accelerate development, improve quality, automate repetitive tasks, and increase delivery efficiency.

Key Responsibilities

Data Platform & Lakehouse Engineering

Design end-to-end Azure data architecture using Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage Gen2, Delta Lake, and Medallion Architecture.
Lead migrations from Informatica PowerCenter and legacy ETL platforms to Azure-native solutions.
Perform migration assessments, complexity analysis, effort estimation, and modernization planning.
Design scalable ingestion, transformation,



and consumption layers following Data Lakehouse best practices.
Implement data modeling, partitioning strategies, schema evolution, and performance optimization techniques.

Data Engineering & Pipeline Development

Build batch and near real-time data pipelines using Databricks, PySpark, Spark SQL, Azure Data Factory, and Synapse.
Design and implement CDC, SCD, UPSERT/MERGE, reconciliation, and data quality frameworks.
Develop reusable ingestion and transformation frameworks to improve delivery consistency and speed.
Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency.

AI-Assisted Engineering & Agentic Development

Leverage GitHub Copilot, Microsoft Copilot, Databricks Assistant, and AI-powered tools to accelerate SQL/PySpark development, ETL pipelines, test data generation, documentation, troubleshooting, and overall engineering productivity.
Develop and utilize AI

📌 Software Engineer I (Hyderabad)
🏢 MetLife
📍 Hyderabad