Area(s) of responsibility

JD, for a Databricks developer

We are looking for an experienced Databricks Developer to design, develop, and maintain scalable data pipelines and analytics solutions using Databricks. The ideal candidate should have solid expertise in Apache Spark, Python, SQL, and cloud-based data engineering technologies to support enterprise data integration and analytics initiatives.

Key Responsibilities
Design, develop, and maintain ETL/ELT pipelines using Databricks.
Develop scalable data processing frameworks using Apache Spark (PySpark).
Build and optimize batch and streaming data pipelines.
Integrate data from multiple sources such as databases, APIs, cloud storage, and enterprise applications.
Implement data transformation, cleansing, and validation processes.
Develop Delta Lake tables and optimize data storage using partitioning and Z-Ordering.
Create reusable notebooks, workflows, and jobs in Databricks.
Monitor and troubleshoot pipeline failures and performance issues.
Collaborate with business analysts, architects,



and stakeholders to understand data requirements.
Follow coding standards, version control, and CI/CD best practices.
Participate in code reviews and technical design discussions.

Technical Skills

Mandatory Skills
Robust experience with Databricks
Expertise in Apache Spark / PySpark
Proficiency in Python
Robust SQL skills
Experience with ETL/ELT development
Experience with Delta Lake
Knowledge of data warehousing concepts
Experience with Git and version control

Preferred Skills
Experience with cloud platforms:
Microsoft Azure – Azure Databricks, ADLS Gen2, Azure Data Factory
Amazon Web Services – S3, Glue, EMR
Google Cloud – BigQuery, Cloud Storage
Experience with streaming technologies such as Spark Structured Streaming.
Knowledge of orchestration tools such as Airflow.
Understanding of data governance and security concepts.

Nice to Have
Experience with Medallion Archit

📌 Azure Subcontractor Hyderabad
🏢 Birlasoft
📍 Hyderabad