Spectral Consultants is Hiring: Senior Data Engineer – Azure Databricks for a leading Accounting & Advisory Firm.

? Location: Bangalore

? Experience: 5–8 Years

We are looking for a Senior Data Engineer with strong hands-on expertise in Azure Databricks, PySpark, SQL and Delta Lake to build scalable, governed data pipelines and analytics/AI-ready datasets.

? Key Responsibilities

- Build scalable ETL/ELT pipelines on Azure Databricks using Medallion Architecture
- Develop Bronze → Silver → Gold → Platinum Delta Lake data models
- Work with PySpark, Spark SQL, Structured Streaming and Auto Loader
- Implement Unity Catalog governance, access controls, lineage and data classification
- Integrate data from ADLS Gen2, AWS, GCP, Snowflake, Redshift, BigQuery and other sources
- Build data-quality, audit, reconciliation and operational-metadata frameworks




- Implement Databricks Workflows/Jobs, DLT/Lakeflow, Git and Databricks Asset Bundles
- Optimize Delta workloads using partitioning, OPTIMIZE, Z-Ordering/Liquid Clustering
- Develop datasets supporting AI/ML, analytics, Power BI and Tableau use cases

? Must-Have Skills

✅ Azure Databricks

✅ PySpark / Spark / Advanced SQL

✅ Delta Lake & Medallion Architecture

✅ Unity Catalog

✅ ADLS Gen2

✅ Structured Streaming / Auto Loader

✅ Databricks Workflows / DLT

✅ Git / CI-CD

✅ Data Quality & Governance

⭐ Valuable to Have

- MLflow / Feature Store / Mosaic AI / Vector Search
- Power BI / Tableau data modelling
- Financial

📌 Senior Data Engineer (Azure Databricks) (Bengaluru)
🏢 Spectral Consultants
📍 Bengaluru