Asrah - Senior Data & AI Engineer (India)

Asrah - Senior Data & AI Engineer (India)

17 Nov
|
Asrah Technologies
|
India

17 Nov

Asrah Technologies

India

Job Summary:

We are looking for a Senior Data Engineer to design, build, and operate our enterprise data platform on Azure and Databricks, with robust hands-on skills in GenAI/LLM application engineering. This role owns the full lifecycle of data pipelines and AI-powered applications from ingestion and orchestration to governance, API development, and deployment.

Key Responsibilities:

GenAI &

- LLM Application Engineering:
- Design and build RAG pipelines, including embedding-based and graph/metadata-grounded retrieval approaches.
- Orchestrate multi-step and multi-agent LLM workflows using LangGraph and LangChain.
- Build and integrate tools/agents using MCP for structured tool-calling and context management.
- Develop backend APIs for AI applications using FastAPI and Python Flask.
- Build and manage LLM/agent workflows using Azure AI Foundry, and implement semantic vector search using Azure AI Search indexes.
- Implement prompt engineering, structured outputs, guardrails, and LLM evaluation/observability practices.

Data Engineering &

- Pipeline Development:
- Design and build scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (PySpark, Spark SQL).
- Develop and maintain Databricks Lakeflow (Delta Live Tables) pipelines across bronze/silver/gold medallion layers.
- Use Databricks Autoloader for incremental, schema-evolving file ingestion into Delta Lake.
- Build reusable, config-driven ingestion frameworks supporting multiple heterogeneous source systems.
- Implement CDC, SCD Type 1/2, and incremental loading strategies.
- Package and deploy Databricks workflows using Databricks Asset Bundles (DABs).

Platform, Infrastructure &

- DevOps:




- Own infrastructure provisioning and deployment using Terraform for Databricks, ADF, and related Azure resources.
- Build and maintain Azure DevOps CI/CD pipelines for automated testing, deployment, and release management.
- Deploy and manage applications on Azure Container Apps, using Docker for containerization.
- Develop and support Databricks Apps for internal data tooling.

Governance, Security &

- Reliability:
- Implement and maintain data governance using Databricks Unity Catalog.
- Design and enforce security policies across managed tables.
- Build monitoring, alerting, and SLA dashboards to ensure pipeline reliability and data quality.
- Perform Spark/Databricks performance tuning, cluster sizing, partitioning, and cost optimization.

Collaboration:

- Partner with data architects, analysts, and business stakeholders to translate requirements into scalable data and AI solutions.
- Document architecture, pipelines, and operational runbooks.
- Mentor junior engineers on Databricks, Python development, and platform engineering best practices.

Preferred Candidate Profile:

- Strong hands-on experience with Azure Data Factory, Databricks (PySpark, Spark SQL, Delta Lake, Autoloader, Lakeflow, DABs), and Unity Catalog.
- Proficiency in Databricks Apps, Azure Container Apps, Docker, and Terraform.
- Strong Python development skills, including API development with FastAPI and Flask.
- Hands-on experience with LLM application engineering (RAG, LangGraph, LangChain, MCP).
- Experience with Azure AI Foundry, Azure AI Search, and vector databases.
- Experience with CI/CD using Azure DevOps and API integration.

📌 Asrah - Senior Data & AI Engineer (India)
🏢 Asrah Technologies
📍 India

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: asrah - senior data & ai engineer (india) / india

Subscribe to this job alert:

Get the latest job offers by email for: asrah - senior data & ai engineer (india) / india