- Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Snowflake.
- Collaborate with cross-functional teams to gather requirements and design solutions for complex business problems.
- Develop high-quality code in Python using Pandas, NumPy, Matplotlib libraries for data manipulation and visualization.
- Ensure scalability, security, and performance of the system by implementing efficient algorithms and optimizing database queries on SQL Server.
- Troubleshoot issues related to pipeline failures or errors in ADF workflows.
Desired Candidate Profile
- 8-12 years of experience in IT industry with expertise in ADF, Databricks, Python programming language.
- Robust understanding of cloud computing concepts with hands-on experience working with Microsoft Azure platform.
- Proficiency in writing complex SQL queries for querying large datasets stored in Snowflake databases.
- Bachelor's degree in Any Specialization (B.Tech / B.E.).