10 Aug
|
Leading
|
Bengaluru
Job Title: Testing Lead – Deep Learning / LLM / VLM
Location
Remote /
Hybrid / On -site
Experience
3–4 years
(hands -on ownership in DL / LLM / GenAI testing)
Employment Type
Full -time
Role Overview
We are
seeking a hands -on Testing Lead to own quality and documentation for our Deep Learning, LLM, and Vision -Language Model (VLM) products. You will
define how we test, measure, document, and communicate AI quality—working
closely with ML, Engineering, and Product teams in a fast -paced startup
setting.
This role
is ideal for someone who believes clear documentation is as critical as good
testing, especially for non -deterministic AI systems.
What You’ll Do
Own Quality & Documentation End -to -End
- Define testing strategy for LLMs,
VLMs, and DL pipelines.
- Create and maintain clear,
lightweight documentation covering:
- Model testing strategies
and assumptions
- Evaluation metrics and
acceptance criteria
- Known limitations, risks,
and failure modes
- Release readiness and
quality sign -off
- Ensure documentation evolves
with models, data, and prompts.
LLM / GenAI Testing
- Design tests for:
- Prompt templates and prompt
changes
- RAG pipelines (retrieval
quality, grounding, hallucination control)
- Multi -turn conversations
and long -context behaviour
- Maintain golden datasets,
regression test suites, and test result summaries.
- Document prompt behaviour,
edge cases, and known model quirks.
Vision & Multimodal Testing
- Test VLMs for image -text
alignment, OCR, captioning, and reasoning.
- Document model performance
across different image types, quality levels, and domains.
- Track and publish model
behaviour changes between versions.
Automation, MLOps & Reporting
- Build Python -based
automation for evaluation and regression testing.
- Integrate tests into CI/CD
and MLOps pipelines.
- Produce readable quality
reports and dashboards for engineers and leadership.
- Monitor and document
production issues such as model/data drift and degradation.
Build a Quality -First Culture
- Establish QA and
documentation standards that scale with a startup.
- Mentor engineers on writing
testable code and meaningful documentation.
- Act as the single source
of truth for AI quality, testing, and known risks.
testing with lead or ownership experience.
DL models, or GenAI systems.
for test automation and data validation.
structured technical documentation.
ambiguity and moving fast in a startup.
multimodal models, or computer vision.
architectures, vector databases, and embeddings.
LangChain, LlamaIndex, MLflow, or similar.
risks, limitations, or compliance requirements.