Role Overview : Define and lead the end-to-end Lakehouse architecture on Databricks ensuring scalability, governance, and performance.
Key Responsibilities
- Design medallion architecture and canonical models.
- Define standards for ingestion, transformation, orchestration, and CI/CD.
- Architect event-driven ingestion and workflows using EventBridge.
- Ensure governance, lineage, and cost optimization.
- Guide Spark/Delta performance best practices.
- Leverage AI copilots for design and documentation.
Must Have Skills
- Expertise in Databricks Lakehouse, Delta Lake, Unity Catalog, and Workflows.
- Strong AWS architecture (S3, IAM, EventBridge).
- Deep Spark performance tuning.
- Strong usage of AI coding copilots (GitHub Copilot, ChatGPT) for code generation, debugging, optimization, and documentation.
Valuable to Have Skills
- Healthcare domain exposure including Quality measures, Risk Adjustment (HCC/RAF), and HEDIS.
- Exposure to FHIR/HL7 data models and patient/member/provider entities.
- HealthLake architecture.
- Streaming architectures