Role:
As one of the founding members of Kanerika's Databricks delivery practice, you will be a hands-on builder responsible for designing, developing, and optimizing data pipelines on Databricks for client engagements. You'll work closely with the Practice Lead to translate architecture into working solutions, build reusable accelerators ahead of client demand, and help establish engineering best practices for the growing team.
Key Responsibilities
Design, build, and optimize ETL/ELT data pipelines on Databricks using PySpark and Spark SQL.
Build and maintain incremental ingestion pipelines using Auto Loader and structured streaming, including checkpoint management and schema evolution handling.
Implement and maintain Delta Lake tables (including Change Data Feed and liquid clustering), Unity Catalog structures, and Databricks Workflows for client and internal projects.
Build reusable accelerators, templates, and demo environments to support pre-sales and speed up future client delivery.
Collaborate with the Practice Lead/Architect on solution design for client engagements,



providing engineering-level input on feasibility and effort.
Perform data quality checks, performance tuning, and cost optimization on Databricks clusters and jobs.
Participate in client-facing technical discussions as needed, including discovery sessions and technical walkthroughs.
Write clean, well-documented, and testable code following team engineering standards.
Mentor junior engineers as the team scales, and contribute to internal knowledge-sharing and best-practice documentation.
Troubleshoot and resolve production data pipeline issues across client environments.
Stay current with Databricks platform releases across Unity Catalog, Lakeflow Declarative Pipelines, and share learnings with the team through internal knowledge sessions and documentation.
Required Skills & Experience
5–9 years of data engineering experience, with at least 2 years of hands-on Databricks experience in production.

📌 Lead Data Engineer (Hyderabad)
🏢 Kanerika
📍 Hyderabad