ABOUT THE ROLE

Build reliable, secure data solutions for healthcare and value-based care. This role owns data engineering, data modelling, data quality and governance, and works with AI engineers to help users understand and work with data. Strong engineering skills, hands-on healthcare data experience and practical knowledge of applying AI to data problems are required.

KEY RESPONSIBILITIES

Data engineering

- Build scalable data pipelines using Azure Databricks, PySpark, Spark SQL and Delta Lake; design and operate lakehouse and Medallion architectures.
- Process claims, eligibility, enrollment, provider records and clinical information from structured and semi-structured sources.
- Handle schema changes, duplicate records, late-arriving data, reprocessing and historical loads; reconcile records and record reasons for rejected data.
- Optimize processing performance, reliability, compute usage and cloud costs. Healthcare data modelling and value based care
- Design reusable models for members, patients, providers, claims, coverage, eligibility and attribution.
- Resolve identities across sources and manage historical changes, overlapping eligibility periods and relationships between entities.
- Define canonical models and maintain source-to-target mappings, transformation rules and version history.
- Build datasets for attribution, PMPM cost and utilization, HCC/RAF risk adjustment, HEDIS and Stars measure inputs, care gaps, readmissions and ED utilization.
- Work with business stakeholders to validate definitions, calculations and reporting outcomes.

Data quality and governance

- Automate checks for completeness, accuracy, consistency, validity and referential integrity,



including healthcare identifiers and code sets such as NPI, ICD-10, CPT/HCPCS, NDC and LOINC.
- Manage rejected records and exceptions with explicit reasons, ownership and resolution processes.
- Maintain data lineage, metadata, catalogues, data contracts and audit trails.
- Implement HIPAA-aligned PHI/PII controls, classification, masking and row- and column-level access management through Unity Catalog.

APPLIED AI AND DATA ENABLEMENT

- Prepare trusted datasets, reference information and knowledge content for AI applications using LLMs, structured outputs, embeddings, retrieval and vector search.
- Collaborate on field mapping, data-quality rule generation and data interpretation; evaluate AI suggestions with representative test datasets and measurable accuracy checks.
- Protect sensitive data in prompts, logs and AI-accessible datasets; enforce read-only access and bounded queries for AI tools.
- Maintain provenance and review records so AI recommendations remain distinguishable from verified data.

ENGINEERING AND DELIVERY

- Write maintainable Python, PySpark and SQL code with meaningful automated tests.
- Manage infrastructure and controlled deployments using Terraform, Databricks Asset Bundles and Git- based CI/CD, including Azure DevOps.
- Monitor production workloads, investigate failures and maintain technical documentation,



design notes and runbooks.
- Validate changes through actual processing runs and reconciled results; explain outcomes clearly to analysts and business stakeholders.

Requirements

REQUIRED SKILLS AND EXPERIENCE

- 5+ years in data engineering, including at least 2 years with Databricks and Apache Spark.
- Strong Python, PySpark, SQL and Spark SQL skills; production experience with Delta Lake, Unity Catalog and Databricks Jobs/Workflows.
- Experience designing data models and delivering lakehouse or Medallion architectures.
- Hands-on healthcare payer or provider data experience, including claims, eligibility/enrollment or clinical data. Healthcare experience is mandatory.
- Working knowledge of value-based care, including attribution, HCC/RAF risk adjustment, quality measures, shared savings, capitation and cost/utilization analysis.
- Experience handling PHI/PII and implementing healthcare privacy, security and access controls.
- Experience with Azure, ADLS Gen2, Entra ID, Key Vault, Terraform, Git and CI/CD.
- Practical experience using LLMs for data tasks and evaluating their outputs.
- Ability to own production delivery, troubleshoot complex issues and communicate with technical and business teams

PREFERRED EXPERIENCE

- X12 EDI experience, including 834, 837 and 835 transactions; HL7 v2, FHIR or ADT feeds.
- LangChain, LangGraph, Databricks Mosaic AI, Model Serving, Vector Search or Genie.
- Entity resolution or master patient index experience; familiarity with Medicare Advantage, Medicaid, MSSP or other accountable-care programs. PostgreSQL, Lakebase or FastAPI experience.

📌 Senior Data Engineer (Hyderabad)
🏢 CittaAI
📍 Hyderabad