Role Summary

We are seeking a highly skilled Azure Data Engineer with a minimum of 5 years of experience in designing, developing, and maintaining modern cloud-based data platforms. The ideal candidate should have strong expertise in Azure Data Factory, Azure Databricks, Spark, SQL, Python, Data Warehousing, and DevOps practices. The role requires hands-on experience in building scalable data pipelines, implementing Lakehouse architectures, and delivering enterprise-grade data solutions.

Key Responsibilities

- Design and develop scalable data ingestion, transformation, and orchestration solutions on Azure.
- Build and maintain batch and streaming data pipelines using Azure Databricks and Azure Data Factory.
- Implement Lakehouse architecture using Delta Lake and Azure Data Lake Storage Gen2.
- Develop high-performance data transformation logic using Spark DataFrame APIs and Spark SQL.
- Optimise data processing workloads through caching, persistence, partitioning, and performance tuning techniques.
- Collaborate with architects, analysts, and business stakeholders to deliver data solutions.




- Implement CI/CD and source control best practices using Git and branching strategies.
- Support production deployments, troubleshooting, and operational excellence.

Mandatory Technical Skills

- Azure Data Factory (ADF): Orchestration, ingestion from on-premises and cloud sources, triggers, linked services, and Integration Runtime (IR).
- Azure Databricks: Data ingestion from ADLS Gen2, Delta Lake, CRUD operations, Time Travel, Optimisation, Merge operations, Streaming, and Auto Loader.
- Apache Spark: Data transformation using DataFrame APIs, Spark SQL, caching, persistence, and performance optimisation.
- Data Warehousing: Solid understanding of Slowly Changing Dimensions (SCD) and data loading strategies.
- SQL: Advanced SQL development and coding expertise. (Window Function/CTE)
- Python: Data handling using Pandas, Object-Oriented Programming concepts, Classes, and Objects.
- DevOps: Git usage, source code management, and branching strategy awareness.

📌 Azure Data Engineer (Hyderabad)
🏢 Cognizant
📍 Hyderabad