We are looking for an experienced Azure Databricks Developer with robust expertise in PySpark and SQL to design, develop, and optimize scalable data solutions on the Azure cloud platform.
Key Responsibilities:
Develop and maintain ETL/ELT pipelines using Azure Databricks and PySpark.
Design, write, and optimize complex SQL queries, stored procedures, and data transformations.
Build scalable data ingestion frameworks from multiple data sources.
Work with large-scale structured and unstructured datasets in a distributed workplace.
Implement data quality checks, performance tuning, and monitoring processes.
Collaborate with business stakeholders and cross-functional teams to understand data requirements.
Integrate Azure services such as ADLS, Azure Data Factory, Azure Synapse, and Azure SQL Database.
Ensure adherence to data governance, security, and best practices.