We are looking for an experienced Azure Databricks Developer with strong expertise in PySpark and SQL to design, develop, and optimize scalable data solutions on the Azure cloud platform.
Key Responsibilities:
- Develop and maintain ETL/ELT pipelines using Azure Databricks and PySpark.
- Design, write, and optimize complex SQL queries, stored procedures, and data transformations.
- Build scalable data ingestion frameworks from multiple data sources.
- Work with large-scale structured and unstructured datasets in a distributed workplace.
- Implement data quality checks, performance tuning, and monitoring processes.
- Collaborate with business stakeholders and cross-functional teams to understand data requirements.
- Integrate Azure services such as ADLS, Azure Data Factory, Azure Synapse, and Azure SQL Database.
- Ensure adherence to data governance, security, and best practices.