Roles and Responsibilities
Design, develop, and maintain scalable, enterprise-grade AI agents and high-volume ELT/ETL pipelines using Python, PySpark, Databricks, Kafka, and FastAPI.
Implement and deploy GenAI agents using Google ADK and Google Flash 2.5+ LLMs to power application automation and workflow optimization via Human-in-the-Loop (HIL) design.
Build and maintain data federation layers for Lambda and Data Mesh architectures using Starburst to enable machine learning, deep learning, and NLP use cases.
Develop, deploy, and automate resilient microservice integrations supporting data-intensive applications on Kubernetes and OpenShift cloud-native platforms.
Integrate agentic AI tools such as Devin.AI and GitHub Copilot via Model Context Protocol (MCP) and advanced prompt engineering to maximize development velocity.
Enforce data quality, security, and risk compliance across the data lifecycle, ensuring adherence to regulatory standards and internal control policies.
Establish CI/CD pipelines, unit testing frameworks,



and engineering best practices to support high-availability platform deployments.

Preferred Candidate Profile

Experience:
Overall Experience: 8+ years in large-scale software engineering or data platform development.
Relevant Experience: 5+ years of hands-on technical lead experience in Python, PySpark, and Databricks pipeline architectures.
Domain Experience: Background in Banking, Financial Services, Retail Products (Cards, Mortgages, Deposits), or Wealth Management risk domains is strongly preferred.

Technical Expertise:
Core Data Engineering: Python, PySpark, Databricks, SQL, Kafka, Data Modeling, Data Warehousing.
AI & Microservices: GenAI Agents, Google ADK, LLM Integrations (Flash 2.5+), MCP, FastAPI, Microservices, Prompt Engineering.
Platform & Infrastructure: Data Mesh, Starburst, OpenShift, Kubernetes, Docker, CI/CD.

📌 Data Engineer Chennai
🏢 SG Analytics
📍 Chennai