19 Sep
|
PwC
|
Bangalore Rural
19 Sep
PwC
Bangalore Rural
Role & responsibilities
- Test and validate LLMbased and RAGbased agents, including reasoning, memory, and decisionmaking behavior
- Design and execute prompt engineering and adversarial testing to uncover hallucinations, bias, and edge cases
- Perform hallucination testing, relevance checks, and factual accuracy validation
- Implement unit tests for LLMs using LLMasaJudge techniques
- Validate agent reasoning chains, memory states, and conversation persistence
- Embed LLMs within testing pipelines to:
- Detect hallucinations
Preferred candidate profile
- LangChain + LangSmith
- DeepEval
- Python (LLM testing harnesses)
- OpenAI / Azure OpenAI APIs
- Git + CI integration
- AWS AgentCore
- AWS CloudWatch / Grafana / Prometheus
- Cursor AI / GitHub Copilot / Claude CLI
Perks and perks
📌 Software Test Engineer (Bangalore Rural)
🏢 PwC
📍 Bangalore Rural