Cortex / Agent Validation Engineer Enterprise AI | Snowflake Cortex | Agentic AI Quality Engineering (Bengaluru)

Cortex / Agent Validation Engineer Enterprise AI | Snowflake Cortex | Agentic AI Quality Engineering (Bengaluru)

17 Nov
|
The Digital Loom
|
Bengaluru

17 Nov

The Digital Loom

Bengaluru

Role Overview We are looking for a skilled Cortex / Agent Validation Engineer to establish and execute quality assurance frameworks for enterprise AI solutions. The role will focus on validating Snowflake Cortex Agents, Cortex Skills, RAG-based applications, and conversational AI solutions. The ideal candidate will develop evaluation frameworks, maintain golden question-answer datasets, define measurable quality benchmarks, and build automated testing and regression processes. You will work closely with AI Engineers, Data Engineers, Product Owners, Business SMEs, and Governance teams to ensure AI solutions deliver accurate, reliable, consistent, and trustworthy results before production release.

Key Responsibilities
• Define AI quality strategies, evaluation metrics, acceptance criteria, and validation thresholds for enterprise AI use cases.
• Create, maintain, version, and manage golden question-answer datasets with expected results and traceability.
• Design and execute functional, regression, factuality, relevance, grounding, consistency, safety, robustness, and performance test scenarios.
• Validate Snowflake Cortex Agents, Cortex Skills, Cortex Analyst, RAG pipelines, conversational AI, and tool-enabled agent workflows.
• Evaluate AI responses for accuracy, hallucinations, source attribution, semantic grounding, confidence scoring, and abstention behaviour.
• Develop automated test frameworks and repeatable evaluation pipelines using Python, SQL, Selenium WebDriver, REST APIs, and CI/CD tools.
• Perform regression testing whenever models, prompts, agents, semantic models, metadata, knowledge sources,



or integrations are updated.
• Identify, analyse, and troubleshoot AI response failures and collaborate with engineering teams to resolve quality issues.
• Prepare test plans, test coverage reports, quality scorecards, defect reports, and production-readiness recommendations.
• Monitor AI quality drift, latency, error patterns, and system reliability.
• Ensure compliance with security, privacy, auditability, and Responsible AI standards. Mandatory Skills and Experience
• 5–8 years of experience in Software Testing, Quality Engineering, Data Validation, AI/ML Validation, or Analytics Engineering.
• Mandatory hands-on experience with Selenium WebDriver for UI and end-to-end test automation.
• Practical experience testing or validating Generative AI, LLMs, Agentic AI, conversational AI, or data-driven applications.
• Experience designing AI evaluation frameworks, quality benchmarks, acceptance thresholds, and golden datasets.
• Solid programming skills in Python and SQL for test automation, data validation, and result analysis.
• Understanding of LLM evaluation concepts, hallucination detection, prompt behaviour, grounding, semantic models, and confidence scoring.
• Experience with REST API testing, Git, CI/CD pipelines,



and Agile testing practices.
• Ability to convert business requirements into measurable test cases, quality metrics, and release acceptance criteria. Preferred / Good-to-Have Skills
• Experience with Snowflake Cortex, Cortex Agents, Cortex Analyst, or Cortex Search.
• Understanding of RAG architectures, vector databases, embeddings, and semantic retrieval.
• Exposure to Azure OpenAI or other enterprise LLM platforms and model-evaluation tools.
• Knowledge of MCP server integrations, LangChain, LlamaIndex, or similar agent frameworks.
• Experience working in banking, financial services, capital markets, wealth management, or other regulated industries.
• Snowflake, cloud, AI/ML, data-quality, or software-testing certifications. Success Measures
• Establish repeatable and auditable AI evaluation frameworks for priority business use cases.
• Ensure AI agents meet agreed accuracy, relevance, grounding, consistency, confidence, and performance benchmarks.
• Build automated Selenium and AI validation coverage to identify functional and behavioural regressions early.
• Identify quality issues and AI response failures with clear root-cause analysis.
• Ensure test results are traceable to approved data sources, business rules, and expected outcomes.
• Provide reliable quality evidence and recommendations for production releases. Working Model
• Hybrid working arrangement.
• Mandatory presence at the Bangalore office 3 days per week. Email
- [email protected]

📌 Cortex / Agent Validation Engineer Enterprise AI | Snowflake Cortex | Agentic AI Quality Engineering (Bengaluru)
🏢 The Digital Loom
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: cortex / agent validation engineer enterprise ai | snowflake cortex | agentic ai quality engineering (bengaluru) / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: cortex / agent validation engineer enterprise ai | snowflake cortex | agentic ai quality engineering (bengaluru) / bengaluru