Role & responsibilities

Key Responsibilities

- Design and implement big-data solutions on Azure Databricks for batch and real-time analytics use

cases.
- Build Spark pipelines that populate downstream data marts, business-intelligence datasets,

data-science feeds, and feature pipelines.
- Process business events from Kafka and persist curated data in ADLS Gen2.
- Design robust lakehouse solutions using Delta Lake, medallion architecture, dimensional modelling,

aggregation, and data-mesh concepts.
- Apply Databricks performance techniques such as liquid clustering, vacuum operations, and

Z-ordering where appropriate.
- Develop orchestration, monitoring, logging, testing, deployment, and recovery capabilities.
- Partner with DevOps teams to establish operationally sound infrastructure and delivery practices.
- Apply data-quality principles including consistency, completeness, accuracy, and lineage.
- Communicate designs and operational considerations to technical and non-technical stakeholders.

Required Skills and Experience
- Bachelors degree or higher in Computer Science, Engineering, Science, or a related discipline.
- 8 to 10 years of data-engineering experience, including 5+ years with Azure cloud-native services.
- Strong Python or PySpark, Spark, SQL, Unix, and Hive experience.
- Hands-on Azure Databricks, Azure Data Factory, ADLS Gen2, Azure SQL Database, and SQL or T-SQL

experience.
- Strong knowledge of Lakehouse,



Delta Lake, Data Mesh, Data Virtualization, dimensional modelling,

and medallion architecture.
- Experience with distributed platforms such as Spark, Kafka, Flink, or HBase and very large data

volumes.
- Knowledge of at least one workflow orchestration framework such as Airflow, Luigi, Oozie, or AWS

Glue.
- Understanding of RDBMS, data lakes, data warehouses, monitoring, logging, and DevOps practices.
- Strong communication skills and experience working in globally distributed teams.

Preferred Skills
- Knowledge of Microsoft Fabric and its framework.
- Experience with Databricks Genie or Datahub.

Confidential recruitment document Page 2
- Experience with Power BI or Tableau.
- Experience with Apache Flink, Apache Kafka, or the ELK stack.

Expected Outcomes
- Deliver scalable, governed, and observable Azure Databricks data products.
- Enable reliable analytical, BI, streaming, and machine-learning data use cases.

Candidate Profile The ideal candidate is collaborative, accountable, quality-focused, comfortable working in a globally distributed setting, and able to communicate clearly with both technical and business stakeholders. Note: This has been structured from the supplied role requirements. Final screening criteria, grade, compensation, work arrangement, and employment terms should be confirmed by the hiring organization.

📌 Azure Databricks Engineer - Pune (Kharadi) - WFO
🏢 CLARITY CONSULTING
📍 Pune