Roles and Responsibilities
Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract insights from various sources.
Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
Develop complex data models using Synapse Analytics and Kubernetes to support real-time analytics workloads.
Implement CI/CD processes for ADF pipelines using Python scripts to automate testing, deployment, and monitoring.
Troubleshoot issues related to pipeline failures or performance problems by analyzing logs, debugging code snippets.