21 Aug
|
Recognized
|
Pune
Immediate joiners required !! 2 days WFO - Hinjewadi Phase 3
Who are we
Fulcrum Digital is an agile and next -generation
digital accelerating company providing digital transformation and technology
services right from ideation to implementation. These services have
applicability across a variety of industries, including banking & financial
services, insurance, retail, higher education, food, healthcare, and
manufacturing.
Key Responsibilities
- Design and execute test cases for LLM agents, RAG pipelines,
agentic workflows, and AI -assisted decision tools
- Validate AI outputs against ground truth using
structured accuracy scoring (NAICS, risk exposure flags, hazard group
mapping)
- Detect hallucinations, reasoning gaps, source fabrication,
and misattributions in model -generated content
- Run multi -model comparative testing across GPT, Claude,
Gemini, and Perplexity — evaluating accuracy, latency, and output
completeness
- Test prompt versions iteratively and track accuracy
changes across prompt cycles
- Validate citation accuracy, document ingestion pipelines,
and cross -document context handling
- Design edge case and negative tests for AI -specific
failure modes — content filter triggers, tool call limits, missing
documents, and incomplete synthesis
- Perform regression testing after model upgrades, prompt
changes, and backend fixes,
and maintain structured QA sign -off in JIRA
- You build evaluation rubrics for non -deterministic outputs and apply LLM -as -a -Judge techniques to score quality at scale