As a Lead Databricks Engineer at KPI Partners supporting Databricks, you will be embedded within customer engagements as a trusted technical advisor. You will apply deep, hands-on Databricks development experience and a strong command of distributed computing on Spark — including Spark runtime internals — to design, build, and optimize production-grade data solutions. You will bring current knowledge across the breadth of the Databricks product and platform, establish CI/CD and MLOps best practices, and work fluently across cloud ecosystems to ensure scalable, performant, and reliable delivery for customers.
KEY RESPONSIBILITIES
- Serve as a lead engineer and trusted technical advisor on customer Databricks engagements, from design through production deployment
- Deliver hands-on development on Databricks across multiple customer projects
- Design and optimize distributed-computing workloads on Spark, applying knowledge of Spark runtime internals
- Implement CI/CD for production deployments
- Operationalize MLOps workflows to support machine-learning use cases
- Advise on and apply the breadth of Databricks product and platform features
- Tune workloads for performance, scalability, and cost efficiency
- Work across two or more cloud ecosystems (AWS, Azure, GCP), applying deep expertise in at least one
- Guide customers through best-practice lakehouse implementation and production readiness
- Collaborate with customer stakeholders and onshore leads overlapping PST hours to ensure timely, high-quality delivery
REQUIRED QUALIFICATIONS & SKILLS
- 8–12 years of experience in data engineering, data platforms, and analytics, including substantial consulting experience.
- Completed the Databricks Data Engineering Qualified certification and the required Databricks classes.
- Minimum 6–8+ projects delivered with hands-on development experience on Databricks.
- Working knowledge of two or more common cloud ecosystems (AWS, Azure, GCP), with deep expertise in at least one.
📌 Senior Data Engineer (Hyderabad)
🏢 KPI Partners
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.