Roles and Responsibilities :

- Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, load data from various sources into Azure storage solutions.
- Collaborate with cross-functional teams to gather requirements for data processing needs and design scalable architectures that meet business needs.
- Develop complex SQL queries to optimize database performance and troubleshoot issues related to data quality, integrity, and security.
- Ensure compliance with industry standards for data governance, privacy, and security by implementing best practices in the development of ADF workflows.

Job Requirements :

- 7-15 years of experience in designing and developing large-scale data pipelines using PySpark on AWS or similar technologies.
- Solid understanding of SQL programming language including query optimization techniques for improved performance.
- Experience with big-data technologies such as Hadoop ecosystem components like Hive & Pig.

📌 Lead Azure Data Engineer (Hyderabad)
🏢 KPI Partners
📍 Hyderabad