Description
We are looking for a Senior Databricks Data Engineer with strong expertise in Databricks, data engineering, and healthcare data. The ideal candidate will be responsible for designing scalable, secure, and high-performance data platforms supporting healthcare analytics and data integration.
The candidate should have strong knowledge of modern data architecture, Lakehouse architecture, SQL, Python, data governance, and healthcare domain workflows.
Responsibilities
- Design and lead Databricks Lakehouse for large-scale healthcare data platforms. - Design data ingestion, transformation, storage, and consumption patterns using Databricks. - Develop Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables/Lakeflow. - Integrate data from multiple healthcare sources, including clinical, claims, eligibility, provider, and patient systems. - Design robust ETL/ELT pipelines using PySpark, SQL, and Databricks. - Establish data quality, validation, reconciliation, lineage, and governance frameworks. - Design secure healthcare data solutions while considering HIPAA and PHI/PII protection requirements. - Work with cloud platforms such as Azure, AWS, or GCP to build scalable data solutions.
- Collaborate with data engineers, data scientists, analysts, product owners, and business stakeholders. - Define technical standards, architectural patterns, and best practices for Databricks development. - Lead performance optimization and cost-management initiatives across the Databricks platform. - Provide technical leadership and mentoring to data engineering teams. - Participate in Agile ceremonies and contribute to technical planning and roadmap discussions.
Qualifications
- Strong hands-on experience with Databricks. - Expertise in Lakehouse architecture and Delta Lake. - Solid PySpark and SQL skills. - Experience with Unity Catalog and data governance. - Experience designing enterprise-scale data platforms. - Strong knowledge of ETL/ELT and data modeling. - Experience with cloud platforms such as Azure, AWS, or GCP. - Experience with data integration and orchestration tools. - Knowledge of CI/CD, Git, and DevOps practices. - Strong understanding of data security and access control. - Experience with Agile/Scrum methodologies.
📌 Data Engineer (Noida)
🏢 EXL
📍 Noida