Roles and Responsibilities
Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Databricks.
Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.
Develop complex SQL queries to optimize database performance and troubleshoot issues.
Implement data transformations using PySpark and DataBricks for big data processing.
Ensure seamless integration of Azure services such as Azure Data Factory (ADF) for productive workflow management.