Role & responsibilities

- Design Lakehouse architectures on Azure Databricks using Delta Lake and Unity Catalog.
- Define medallion (Bronze/Silver/Gold) patterns, data models, and enterprise data products.
- Architect ingestion frameworks (batch, streaming, CDC) using Auto Loader, ADF, Event Hubs, Kafka.
- Establish governance, security, and RBAC via Unity Catalog, Key Vault, and service principals.
- Optimize performance and cost (cluster sizing, Photon, Liquid Clustering, Z-Order, caching).
- Lead CI/CD with Databricks Asset Bundles, Git, and Azure DevOps / GitHub Actions.
- Guide teams on DLT, MLflow, Delta Sharing, and Lakehouse Monitoring.
- Partner with Cyber, Cloud, and IAM teams on approvals, allow-listing, and compliance.
- Mentor data engineers and review designs, code,



and production readiness.

Must-Have Skills

- Deep expertise in Azure Databricks, Delta Lake, Unity Catalog, Spark (PySpark/SQL).
- Robust on Azure services: ADLS Gen2, ADF, Synapse, Key Vault, Entra ID, Event Hubs, Functions.
- Proven experience designing medallion / lakehouse architectures at enterprise scale.
- Solid grounding in data modeling (dimensional, Data Vault, or 3NF) and SQL performance tuning.
- CI/CD, IaC (Terraform/Bicep), Git-based workflows, and DevOps practices.
- Data governance, lineage, security, and privacy controls in regulated environments

📌 Data Engineer (Ahmedabad)
🏢 EY
📍 Ahmedabad