- Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to integrate various data sources into a centralized repository.
- Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business objectives.
- Develop complex SQL queries to extract insights from large datasets stored in Oracle databases.
- Troubleshoot issues related to ADF pipeline failures and optimize performance for improved efficiency.
Job Requirements :
- 6-10 years of experience in Data Engineering with expertise in Azure Data Factory (ADF).
- Robust understanding of Microsoft Azure services including storage, compute, networking etc.
- Proficiency in working with Oracle databases and ability to write complex SQL queries.
- Experience with big data technologies such as Hadoop/Hive/Pyspark/Data Bricks.