Purpose/Objective

Databricks Engineer will lead the design, development and operations of scalable data pipelines and Lakehouse solutions on Databricks.

The role will be responsible for translating business and platform requirements into reliable, secure, governed and cost-efficient data solutions.

Key Responsibilities of Role

Key Responsibilities - Lead Databricks-based data engineering delivery for batch, streaming and analytical use cases.

- Design and implement data pipelines using Python, PySpark, SQL, Delta Lake and Databricks workflows.
- Implement data quality, error handling, logging, lineage, metadata and operational monitoring controls.
- Work with architects to define Lakehouse patterns, medallion architecture, reusable frameworks and data product standards.
- Support Unity Catalog, access control, data governance, security and compliance implementation.
- Optimise Databricks jobs, clusters, SQL queries and storage patterns for performance and cost.
- Coordinate with business, analytics, AI, cloud, cybersecurity and operations teams to deliver production-grade solutions.
- Review code, guide engineers, resolve technical blockers and ensure delivery quality.
- Maintain documentation, design artefacts, runbooks and operational handover material. Required Skills - Strong hands-on Databricks development experience.

- Robust Python and PySpark coding skills.
- Good SQL and data modelling knowledge.
- Experience with Delta Lake,



Databricks Jobs/Workflows and performance optimisation.
- Knowledge of data ingestion, transformation, orchestration and production support practices.
- Understanding of cloud storage, IAM, networking and security basics for data platforms.
- Ability to lead small teams and coordinate delivery across multiple stakeholders.

Preferred Skills - Unity Catalog and Databricks governance experience.

- Delta Live Tables, structured streaming and real-time data processing exposure.
- CI/CD for data engineering using GitHub, Azure DevOps or Databricks Asset Bundles.
- Knowledge of data mesh, data products, data quality frameworks and MLOps/DataOps.
- Databricks certification will be an added advantage.

Success Measures - Stable, reusable and well-governed Databricks pipelines in production.

- Improved data quality, lineage, auditability and platform reliability.
- Reduction in pipeline failures, rework and manual operational intervention.
- Improved performance and cost efficiency of Databricks workloads.
- Better adoption of standardised Lakehouse and data engineering patterns.

Technical Competencies Databricks & Lakehouse Architecture,Data Engineering & Pipeline Development,Data Governance, Security & Operational Excellence,Performance Optimisation & Platform Engineering

Qualifications and Experience

3-5 years overall experience, with strong hands-on Databricks and data engineering background

📌 Assistant Manager - Databricks (Ahmedabad)
🏢 Adani Group
📍 Ahmedabad