Roles and Responsibilities
- Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Collaborate with cross-functional teams to gather requirements and design solutions for complex data processing needs.
- Develop high-quality code in Python using PySpark, Azure Databricks, and other relevant technologies.
- Troubleshoot issues related to ADF workflows, Databricks clusters, and SQL queries.
- Ensure compliance with security standards by implementing access controls on datasets and resources.
Desired Candidate Profile
- 5-8 years of experience as an Azure Data Engineer or similar role.
- Solid understanding of Azure services such as Data Factory (ADF), Databricks, Storage Accounts, Cosmos DB etc. .
- Proficiency in programming languages like Python; hands-on experience with PySpark is required.
- Bachelor's degree in Any Specialization (B.Tech / B.E.).