Responsibilities

MSA Role: Lead Software Engineer – Databricks (Mosaic AI)
Experience: 8–11 Years
- Lead design and development of GenAI‑enabled data and AI solutions using Databricks, Mosaic AI, LLMs, RAG pipelines, and AI agents.
- Own end‑to‑end architecture and implementation of scalable, production‑grade data pipelines using Databricks (PySpark, SQL, Delta Lake).
- Design, implement, and govern Delta Live Tables (DLT) pipelines for trusted, auditable, and resilient data processing.
- Define and enforce Unity Catalog–based data and AI governance, including catalogs, schemas, access control, lineage, and ownership.
- Architect and manage datasets, features, and views to support LLM inference, embeddings, RAG, and AI agent workflows.
- Design and integrate LLM‑based capabilities (chat, Q&A;, summarization, automation) using Azure OpenAI / OpenAI.
- Lead usage of vector databases and embeddings to build scalable search, retrieval, and RAG solutions.
- Drive experiment tracking, lineage, and lifecycle management using MLflow integrated with Unity Catalog.




- Ensure robust data quality, observability, performance, reliability, and cost optimization across Databricks workloads.
- Establish monitoring, alerting, and operational best practices for data and AI pipelines.
- Collaborate closely with architects, data scientists, product owners, and business stakeholders to shape GenAI solution design.
- Translate business objectives into robust Databricks and Mosaic AI technical solutions.
- Own production stability, incident response, root‑cause analysis, and continuous improvement.
- Lead code reviews and enforce Databricks, Spark, and GenAI engineering best practices.
- Mentor Specialist and Junior engineers; provide technical direction and hands‑on guidance.
- Plan, estimate, and deliver work aligned with Agile delivery practices and sprint goals.
- Support global collaboration and be flexible to align with Paris business hours when required.
- Own delivery of GenAI feat

📌 Lead Software Engineer (Bengaluru)
🏢 Société Générale
📍 Bengaluru