Job Description

Spectral Consultants is currently hiring for a Senior Data Engineer – Azure Databricks for a leading Professional Services & Accounting organization.

n

n

This is an opportunity to work on contemporary Lakehouse platforms and next-generation AI & Analytics initiatives, building scalable, governed, and AI-ready data solutions using Azure Databricks and the broader Azure ecosystem.

n

n

Location: Bangalore

n

Experience: 5–9 Years

n

Work Mode: Hybrid

n

n

About the Role:-

n

n

As a Senior Data Engineer, you will design and develop scalable ETL/ELT pipelines and curated Lakehouse data models using Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, and ADLS Gen2. You will collaborate with Analytics, AI/ML, Product, and Business teams to deliver reliable, governed, and AI-ready datasets.

n

n

Key Responsibilities:-

n

n
- Design and maintain scalable ETL/ELT pipelines using Azure Databricks.
n
- Build data pipelines using Medallion Architecture (Bronze, Silver, Gold).
n
- Develop and optimize Delta Lake data models for Analytics and AI/ML use cases.
n
- Implement data ingestion using Spark Structured Streaming, Auto Loader, connectors, and Delta Sharing.
n
- Manage governed data using Unity Catalog and integrate with ADLS Gen2.
n




- Implement data quality, lineage, audit, reconciliation, and governance frameworks.
n
- Orchestrate workflows using Databricks Jobs/Workflows and DLT/Lakeflow.
n
- Implement CI/CD using Git and Databricks Asset Bundles.
n
- Troubleshoot production issues and collaborate with cross-functional teams to deliver reliable, consumption-ready datasets.
n

n

n

Must-Have Skills:-

n

n
- 5–8 years of hands-on experience in Data Engineering/Data Integration/Analytics Engineering.
n
- Strong expertise in Azure Databricks, PySpark/Spark, and SQL.
n
- Hands-on experience with Delta Lake and Medallion Architecture.
n
- Strong experience with Unity Catalog and ADLS Gen2.
n
- Experience building scalable ETL/ELT pipelines and Databricks Jobs/Workflows.
n
- Strong understanding of Data Governance, Data Quality, Lineage, Access Control, and PII handling.
n
- Experience with Git, CI/CD, and modern data engineering practices.
n
- Strong analytical, problem-solving, and stakeholder management skills.
n

n

n

Important Eligibility: Only Immediate Joiners or candidates whose Last Working Day (LWD) falls in September are eligible to apply.

📌 Senior Data Engineer-(Immediate Joiners with LWD (Sept Month) Only Apply! (Bengaluru)
🏢 Spectral Consultants
📍 Bengaluru