We are looking for an experienced Azure Databricks Developer with robust expertise in Azure cloud technologies, data engineering, and big data processing. The ideal candidate should have hands-on experience in designing, developing, and optimizing scalable data pipelines using Azure Databricks and associated Azure services. The requirements are aligned with internal Azure Databricks engineering and Azure Data Engineering job descriptions.
Mandatory Skills
- Azure Databricks
- PySpark / Spark
- Azure Data Factory (ADF)
- Azure Data Lake Storage (ADLS Gen2)
- SQL
- Delta Lake
- Azure DevOps / CI-CD
- Data Warehousing concepts
- ETL/ELT Development
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and Azure Data Factory.
- Develop data ingestion, transformation, and processing solutions for large-scale datasets.
- Build and optimize Spark/PySpark jobs for data processing and analytics workloads.
- Work with Azure services such as ADLS, Azure SQL, Event Hub, Azure Functions, Synapse, and Databricks.
- Implement data quality checks, monitoring, and performance tuning of data pipelines.
- Collaborate with business and technical stakeholders to gather requirements and deliver data solutions.
- Support CI/CD implementation and deployment automation using Azure DevOps.
Preferred Skills
- Structured Streaming
- Kafka/Event Hubs
- Unity Catalog
- Azure Purview
- Data Governance and Lineage
- Snowflake or Synapse Analytics
- Experience with cloud migration projects
If you are comfortable with the above and are available for the virtual interview on 17th September, please apply for this opportunity. Once you confirm your interest, I will connect with you for the required data collection and further interview process.