Roles and Responsibilities :

- Design, develop, and maintain complex data pipelines using Python to extract insights from large datasets.
- Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
- Develop and implement machine learning models using retrieval augmented generation techniques to improve data quality and accuracy.
- Troubleshoot issues with existing systems, identifying root causes and implementing fixes to ensure optimal performance.

Job Requirements :

- 5 - 8 years of overall experience in software engineering, data science, or AI/ML development, with at least 3+ years focused on AI/LLM/GenAI testing or agent-based systems.
- Python expertise in scripting, automation, and debugging.
- Strong PySpark experience in distributed testing, data validation, and pipeline testing.
- Hands-on knowledge of GenAI concepts, including LLMs, prompting, context management, RAG pipelines, agent tool-calling, and multi-agent orchestration.
- Experience with agent development and deployment frameworks such as LangGraph,



AutoGen, CrewAI, Copilot Studio Agent SDK, and Vertex AI/OpenAI agent frameworks.

- Solid understanding of agent architecture covering skills, tools, connectors, memory, guardrails, and observability.

- Familiarity with up-to-date GenAI/agent evaluation frameworks such as Langsmith evaluation, AutoGen agent-behavior assessment utilities etc. for benchmarking reliability, grounding, tool-use correctness, and multi-agent performance.
- Strong foundation in functional and regression testing, scenario and edge-case testing, LLM safety and hallucination testing, and workflow validation.
- Experience creating evaluation datasets and defining success criteria for AI behavior.
- Good understanding of API testing frameworks, data engineering concepts, cloud workflow execution (Azure/GCP/AWS), and CI/CD pipelines for test automation.

Excellent communication skills for cross-functional collaboration. .

📌 Senior Analyst-GenAI (Hyderabad)
🏢 Naukri Assist
📍 Hyderabad