Job Description - Data Engineer (Azure | Databricks) + AI Engineering Exposure

Role Title :Data Engineer (Azure Databricks) Engagement Type :Contract (Vendor Hiring) Experience Range :3-8 years Location :India Team :IDO CIS - BI & Analytics

Role Overview

We are looking for a contract Data Engineer to support enterprise BI & Analytics deliveries within IDO. The role will design, build, and operate scalable data pipelines and curated datasets on Microsoft Azure , with strong hands-on delivery in Databricks (Spark/Delta) . Exposure to AI engineering / GenAI enablement is a robust plus for future roadmap initiatives.

Key Responsibilities

- Build and maintain data pipelines on Azure (ingestion, transformation, validation, and publishing) using Databricks and Azure services.
- Develop curated, analytics-ready datasets for BI/reporting and downstream consumers.
- Good understanding of FACT and dimensional modelling.
- Implement incremental load patterns, backfills, and handling of late-arriving data to minimize full reloads.
- Ensure robust data quality , reconciliation checks, and pipeline monitoring/alerting.
- Optimize performance and cost (Spark tuning, partitioning strategies, file formats).




- Maintain documentation (source-to-target mappings, runbooks) and support releases/production issues in an Agile model.

Must-Have Skills (Azure Data Engineering + Databricks)

- 3-8 years of hands-on Data Engineering experience.

- Strong experience with Azure data ecosystem and Databricks :

- Spark (PySpark), Delta Lake (MERGE/upserts, schema evolution), Lakehouse patterns

- Strong SQL and Python (production-grade coding).

- Orchestration experience with Azure Data Factory and/or Databricks Workflows .

- Practical experience implementing incremental processing and robust operational pipelines (retry, idempotency, monitoring).

Good-to-Have Skills (AI Engineering / Future-Forward)

- Exposure to ML/AI enablement (feature datasets, ML lifecycle awareness, reproducible training data).
- Familiarity with GenAI/RAG data preparation (document ingestion, chunking, metadata enrichment, embeddings).
- Understanding of vector search/vector databases concepts (hands-on is a plus).
- Awareness of governance and secure data handling (PII, access control).

📌 De&a - Core - Advanced Visualization & Cloud Reporting - Azure Analysis Services (Pune)
🏢 Zensar Technologies
📍 Pune