Azure Databricks, PySpark, Azure Data Factory, SQL, Microsoft Azure, Public Cloud, Terraform, ARM Templates, Azure DevOps, Infrastructure as Code (IaC), DevOps, Cloud Security, Security Testing
We are looking for an experienced Azure Databricks Data Engineer with strong expertise in Azure Cloud, Databricks and PySpark. The candidate will work on designing, developing and supporting cloud-based data solutions and Azure deployments.
Core Skills
- Strong experience with Azure Databricks and PySpark
- Experience working as a Data Engineer on Azure Cloud
- Good knowledge of Azure Data Factory and SQL
- Experience with public cloud environments
- Strong understanding of Infrastructure as Code
- Hands-on experience with Terraform and ARM Templates
- Experience with Azure DevOps and DevOps practices
- Knowledge of security hardening and testing
- Robust technical documentation and communication skills
Key Responsibilities
- Design and develop Azure cloud solutions using Databricks and PySpark.
- Work with Azure Data Factory and SQL for data engineering requirements.
- Lead and support implementation of Azure infrastructure using Infrastructure as Code.
- Automate standardized Azure deployments using Azure DevOps, Terraform, ARM templates and other deployment technologies.
- Provide technical support and troubleshoot Azure-related issues.
- Gather business requirements and translate them into appropriate technical solutions.
- Maintain documentation related to design, development, build and deployment methods.
- Provide guidance and assistance to support teams.
- Support security hardening and security testing activities.