Project description

Designed and implemented a scalable enterprise Lakehouse platform on Azure using Azure Data Factory, Delta Lake, ADLS Gen2, and Databricks. Migrated legacy ETL processes to cloud-native ELT pipelines supporting both batch and near real-time workloads. Established reusable data frameworks, metadata-driven orchestration, data quality validation, and automated CI/CD pipelines.

Responsibilities

Designed and implemented scalable data pipelines using Azure Data Factory, Databricks, Delta Lake, Python, and SQL.

Architected Lakehouse solutions supporting enterprise analytics, AI/ML, and Generative AI workloads.

Developed structured and unstructured data pipelines for RAG, vector search, and AI-powered applications.

Built reusable frameworks, improving code quality, scalability, and development productivity.

Optimized ETL/ELT processes, reducing data latency and improving performance.

Implemented CI/CD, automated testing, monitoring, and DataOps best practices.

Established data quality,



governance, security, and metadata management standards.

Collaborated with cross-functional teams to deliver business-driven data and AI solutions.

Mentored engineers and led technical design and architecture discussions.

Drove cloud modernization initiatives, improving reliability, scalability, and operational efficiency.

Skills

Must have

5+ years of experience

Azure Data Factory (ADF)

Databricks

Delta Lake

Python

SQL / PL-SQL

PySpark

Azure Data Lake (ADLS)

CI/CD (Azure DevOps)

Data Modeling

ETL/ELT Pipelines

Nice to have

Azure AI Search

Microsoft Fabric

Kafka

Snowflake

Terraform

Docker / Kubernetes

Microsoft Purview

Other

Languages

English: C1 Advanced

Seniority

Regular

Remote India, India

Req. VR-124208

AI/ML

Cross Industry Solutions

29/07/2026

Req. VR-124208

📌 Azure AI Data Engineer (India)
🏢 Luxoft
📍 India