Country/Region: IN

Requisition ID: 38601

Work Model:

Position Type:

Salary Range:

Location: INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI

Title: Azure Databricks Specialist

Description:

Area(s) of responsibility

Skills: Azure Databricks Engineer

Experience: 4–7 Years
Location: Pune, Mumbai, Noida, Chennai, Bangalore

Role Summary

We are looking for a highly skilled Databricks Data Engineer with strong hands‑on expertise in Databricks, PySpark, ADF, and SQL. The candidate will be responsible for architecting, developing, and optimizing up-to-date data platforms and analytical solutions on Azure. The role requires strong experience building enterprise-grade data pipelines, enabling ingestion from diverse sources, implementing complex transformations, and supporting scalable analytics initiatives.

Key Responsibilities

- Data Engineering Execution

- Build and maintain ingestion frameworks (ADF / Databricks / Spark)
Implement Bronze -> Silver transformations aligned to architecture
Architect data quality checks, schema validation, and contract rules
- Build and optimize robust, high-throughput ELT/ETL pipelines, enabling ingestion, transformation, and curation of structured, semi‑structured, and unstructured data.
- Integrate data from multiple on‑premise and cloud‑based systems, APIs, and third-party sources.
- Implement complex transformations using PySpark, ensuring performance efficiency and code modularity.
- Build orchestration workflows in ADF, including pipelines, triggers, linked services, integration runtimes, and parameterized datasets.
- Familiar with using Databicks Genie.

2. Databricks & PySpark Engineering





- Develop scalable transformation scripts using PySpark on Databricks, applying advanced optimizations like caching, partitioning, and Delta Lake capabilities.
- Implement Delta Lake features—ACID transactions, schema enforcement, schema evolution, and time travel—across the data lifecycle.
- Perform performance tuning, handling bottlenecks related to cluster configuration, shuffle operations, joins, and parallelization.
- Collaborate with platform teams to manage Databricks clusters, jobs, notebooks, and CI/CD integrations.

4. Collaboration & Stakeholder Management

- Collaborate closely with customer IT and business teams to understand data requirements and deliver reliable, production-ready solutions.
- Work with architects, product owners, and cross-functional engineering teams to align technical delivery with business objectives.
- Provide guidance and mentoring to junior engineers when required.

Mandatory Skills & Experience

- Minimum 4+ years of experience in data engineering, with strong hands‑on exposure to Azure data ecosystem.
- At least 3 years of real project experience in Databricks (Azure Data Platform (ADF, ADLS, Azure SQL) & Databricks (DLT, Delta Lake, Spark, Workflows, Unity Catalog).
- At least 2 years of hands-on experience building data pipelines using Azure Data Factory (ADF).
- At least 2 years of experience developing PySpark-based transformations in Databricks.
- Strong SQL programming experience, including writing complex queries, performance tuning, and handling large datasets.

Knowledge of CI/CD pipelines (Azure DevOps preferred) for automated deployment of ADF/Databricks artifacts

📌 Azure Databricks Specialist (Hinjawadi)
🏢 Birlasoft
📍 Hinjawadi