Roles and Responsibilities :
Design, develop, and maintain complex data pipelines using Python to extract insights from large datasets.
Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
Develop and implement machine learning models using retrieval augmented generation techniques to improve data quality and accuracy.
Troubleshoot issues with existing systems, identifying root causes and implementing fixes to ensure optimal performance.

Job Requirements :
5 - 8 years of overall experience in software engineering, data science, or AI/ML development, with at least 3+ years focused on AI/LLM/GenAI testing or agent-based systems.
Python expertise in scripting, automation, and debugging.
Robust PySpark experience in distributed testing, data validation, and pipeline testing.
Hands-on knowledge of GenAI concepts, including LLMs, prompting, context management, RAG pipelines, agent tool-calling, and multi-agent orchestration.
Experience with agent development and deployment frameworks such as LangGraph,



AutoGen, CrewAI, Copilot Studio Agent SDK, and Vertex AI/OpenAI agent frameworks.
Solid understanding of agent architecture covering skills, tools, connectors, memory, guardrails, and observability.
Familiarity with up-to-date GenAI/agent evaluation frameworks such as Langsmith evaluation, AutoGen agent-behavior assessment utilities etc. for benchmarking reliability, grounding, tool-use correctness, and multi-agent performance.
Solid foundation in functional and regression testing, scenario and edge-case testing, LLM safety and hallucination testing, and workflow validation.
Experience creating evaluation datasets and defining success criteria for AI behavior.
Good understanding of API testing frameworks, data engineering concepts, cloud workflow execution (Azure/GCP/AWS), and CI/CD pipelines for test automation.

Excellent communication skills for cross-functional collaboration. .

📌 Senior Analyst Genai Chennai (India)
🏢 Naukri Assist
📍 India