Roles and Responsibilities

- Design, develop, and maintain scalable, enterprise-grade AI agents and high-volume ELT/ETL pipelines using Python, PySpark, Databricks, Kafka, and FastAPI.
- Implement and deploy GenAI agents using Google ADK and Google Flash 2.5+ LLMs to power application automation and workflow optimization via Human-in-the-Loop (HIL) design.
- Build and maintain data federation layers for Lambda and Data Mesh architectures using Starburst to enable machine learning, deep learning, and NLP use cases.
- Develop, deploy, and automate resilient microservice integrations supporting data-intensive applications on Kubernetes and OpenShift cloud-native platforms.
- Integrate agentic AI tools such as Devin.AI and GitHub Copilot via Model Context Protocol (MCP) and advanced prompt engineering to maximize development velocity.
- Enforce data quality, security, and risk compliance across the data lifecycle, ensuring adherence to regulatory standards and internal control policies.
- Establish CI/CD pipelines, unit testing frameworks,



and engineering best practices to support high-availability platform deployments.

Preferred Candidate Profile

Experience:

- Overall Experience: 8+ years in large-scale software engineering or data platform development.
- Relevant Experience: 5+ years of hands-on technical lead experience in Python, PySpark, and Databricks pipeline architectures.
- Domain Experience: Background in Banking, Financial Services, Retail Products (Cards, Mortgages, Deposits), or Wealth Management risk domains is strongly preferred.

Technical Expertise:

- Core Data Engineering: Python, PySpark, Databricks, SQL, Kafka, Data Modeling, Data Warehousing.
- AI & Microservices: GenAI Agents, Google ADK, LLM Integrations (Flash 2.5+), MCP, FastAPI, Microservices, Prompt Engineering.
- Platform & Infrastructure: Data Mesh, Starburst, OpenShift, Kubernetes, Docker, CI/CD.

📌 Data Engineer (Pune)
🏢 SG Analytics
📍 Pune