As a Lead Databricks Engineer at KPI Partners supporting Databricks, you will be embedded within customer engagements as a trusted technical advisor. You will apply deep, hands-on Databricks development experience and a strong command of distributed computing on Spark — including Spark runtime internals — to design, build, and optimize production-grade data solutions. You will bring current knowledge across the breadth of the Databricks product and platform, establish CI/CD and MLOps best practices, and work fluently across cloud ecosystems to ensure scalable, performant, and reliable delivery for customers.
KEY RESPONSIBILITIES
• Serve as a lead engineer and trusted technical advisor on customer Databricks engagements, from design through production deployment
• Deliver hands-on development on Databricks across multiple customer projects
• Design and optimize distributed-computing workloads on Spark, applying knowledge of Spark runtime internals
• Implement CI/CD for production deployments
• Operationalize MLOps workflows to support machine-learning use cases
• Advise on and apply the breadth of Databricks product and platform features
• Tune workloads for performance, scalability, and cost efficiency
• Work across two or more cloud ecosystems (AWS, Azure, GCP), applying deep expertise in at least one
• Guide customers through best-practice lakehouse implementation and production readiness
• Collaborate with customer stakeholders and onshore leads overlapping PST hours to ensure timely, high-quality delivery REQUIRED QUALIFICATIONS & SKILLS
• 8–12 years of experience in data engineering, data platforms, and analytics, including substantial consulting experience.
• Completed the Databricks Data Engineering Skilled certification and the required Databricks classes.
• Minimum 6–8+ projects delivered with hands-on development experience on Databricks.
• Working knowledge of two or more common cloud ecosystems (AWS, Azure, GCP), with deep expertise in at least one.
📌 Senior Data Engineer (Hyderabad)
🏢 KPI Partners
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.