Job Description

We're Hiring | Databricks Data Engineer | Pan India | TCS

n

We are looking for experienced Databricks Data Engineers to join our growing team. The ideal candidate should have strong hands-on experience in PySpark/Spark SQL engineering, Delta Lake, and Unity Catalog governance to build reliable, high-performance, production-grade data products for BFSI modernisation, migration, and AI-ready data platform programs.

n

Location

n

Pan India

n

Experience

n

5–10 Years (role-dependent), including 3+ years hands-on Databricks

n

Employment Type

n

Full-time

n

Key Responsibilities

n

n
- Strong experience and understanding of Data Warehouse and Data Lake platform architecture and design.
n
- Design and develop scalable end-to-end ETL/ELT pipelines on Azure/AWS Databricks using PySpark, Spark SQL, and Delta Lake.
n
- Implement medallion (Bronze–Silver–Gold) Lakehouse architectures with automated ingestion, schema evolution, and performance optimisation.
n
- Build streaming and batch ingestion using Auto Loader, Lakeflow/Structured Streaming, Delta Live Tables (DLT), and Azure Data Factory.
n
- Enforce data governance, lineage, access control, and cataloguing through Unity Catalog, RBAC, and Delta Sharing.
n
- Optimise Spark workloads for cost and performance (partitioning, caching,



cluster sizing, query tuning) across large datasets.
n
- Support legacy-to-Lakehouse migrations (Hadoop/DataStage/RDBMS) leveraging accelerators such as Lakebridge and reusable frameworks.
n
- Establish CI/CD for Databricks assets using Azure DevOps/Git, and build automated data validation and testing utilities.
n
- Collaborate with architects, AI/ML specialists, and client teams on solutioning, demos, PoCs, and production rollouts.
n

n

Mandatory Skills

n

n
- Azure/AWS Databricks (Notebooks, Workflows, Jobs)
n
- PySpark, Spark SQL, Python, SQL
n
- Delta Lake, Delta Live Tables (DLT), Auto Loader
n
- Unity Catalog, RBAC, data governance & lineage
n
- Medallion/Lakehouse architecture design
n
- ETL/ELT pipeline engineering & performance tuning
n

n

Positive to Have

n

n
- Azure Data Factory (ADF), ADLS Gen2, AWS Glue
n
- Apache Airflow orchestration, DBT, streaming
n
- CI/CD — Azure DevOps, Git; DataOps practices
n
- Data migration & modernisation (Hadoop/RDBMS)
n
- Power BI/analytics enablement, Lakebase
n
- Exposure to GenAI/Agentic AI on Databricks
n

n

Your Next Career Opportunity Awaits!

n

Apply now and get a faster recruitment process.

n

Click here to apply

📌 Databricks Data Engineer (Karnataka)
🏢 Berribot
📍 Karnataka