Cloud Azure Databricks Data Engineer

Role Overview:

We are seeking a skilled and experienced Cloud Azure Databricks Data Engineer to join our agile team. The ideal candidate will have extensive experience in building, managing, and optimizing data pipelines and solutions on Azure Databricks. They should also possess strong expertise in PySpark, Python, SQL, Unity Catalog, and excellent communication skills.

Key Responsibilities
Design, develop, and maintain scalable data pipelines and workflows on Azure Databricks.
Delivery lead from onsite for all client communication and escalations.
Utilize PySpark and Python for data processing, transformation, and integration.
Develop and optimize SQL queries to enable productive data retrieval and analysis.
Implement and manage Unity Catalog for data governance, access control, and metadata management.
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
Optimize data pipelines for performance, scalability, and cost-effectiveness.
Ensure the security,



compliance, and integrity of data.
Troubleshoot and resolve issues related to data pipelines and processes.
Create and maintain documentation for data workflows, processes, and solutions.
Communicate effectively with stakeholders to provide project updates and insights.

Required Skills and Qualifications
5+ years of experience in data engineering or a related field.
Robust hands-on experience with Azure Databricks.
Proficiency in PySpark and Python for large-scale data processing.
In-depth knowledge of SQL for data analysis and transformation.
Experience with Unity Catalog for managing data governance and access control.
Familiarity with Azure cloud services such as Azure Data Lake, Azure Data Factory, and Azure Synapse Analytics.
Understanding of data modelling concepts and ETL/ELT workflows.
Excellent problem-solving and analytical skills.
Strong communication skills

📌 Azure Databricks Data Engineer Noida (India)
🏢 Topcoder
📍 India