Role description

Lead Databricks Developer (Offshore)

Looking for 2 robust Lead Databricks Developers with streaming expertise.

Key Responsibilities

- Design and develop end-to-end data pipelines using Databricks (PySpark, Delta Lake).

- Build real-time streaming ingestion pipelines from Kafka and other event-driven sources.

- Implement layered data architecture (Bronze, Silver, Gold) to support transformation and reporting.

- Develop data processing logic for transactional data ingestion, validation, and enrichment.
- Apply data quality, deduplication, and business rules for healthcare datasets.

- Support cloud migration from on-premise systems to Databricks + Snowflake ecosystem.

- Optimize pipeline performance to meet SLA-driven processing timelines.
- Implement and support CI/CD pipelines and DevOps integration.
- Collaborate with business and technical stakeholders to deliver scalable solutions.

Required Skills

- Strong expertise in Databricks (Streaming & Batch)

- Advanced proficiency in PySpark / Spark SQL/Spark Streaming

- Experience in real-time data streaming (Kafka)

- Strong understanding of data modeling and ETL/ELT pipeline design

- Hands-on experience with AWS (S3, Airflow, Lambda)

- Experience with CI/CD, Git, and deployment automation

Domain Experience

- Experience in Healthcare Payer domain (Claims, Membership,



Coverage)

- Exposure to FHIR / healthcare data standards preferred
- Understanding of data privacy, security, and governance requirements (PII/HIPAA)

Leadership Expectations

- Operate at a Lead level driving design, development, and delivery

- Mentor team members and ensure delivery quality

- Own modules independently and collaborate across distributed teams

Experience

- 8+ years in Data Engineering

- 3+ years hands-on experience with Databricks

- Prior experience in Tech Lead/Senior roles

Looking for 2 strong Lead Databricks Developers with streaming expertise

Skills

databricks,spark sql,healthcare,apache spark,apache kafka,delta lake,

About UST
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.

📌 Lead II - Databricks Engineer (India)
🏢 UST
📍 India