- Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure Databricks.
- Collaborate with cross-functional teams to gather requirements for data warehousing solutions and implement ETL processes that meet business needs.
- Develop complex SQL queries to optimize database performance and troubleshoot issues in existing databases.
- Ensure high availability of data systems by implementing monitoring tools and performing regular maintenance tasks.
Job Requirements :
- 10-15 years of experience in IT Services & Consulting industry as a Data Engineer or similar role.
- Robust expertise in Azure Data Factory (ADF), Azure Databricks, SQL, and data warehousing concepts.
- Experience working on large-scale projects involving big data processing using Hadoop technologies such as Spark.