Role
Overview

As a Test
Automation Lead at Dailoqa, you’ll architect and implement robust testing
frameworks for both software and AI/ML systems. You’ll bridge the gap between traditional
QA and AI-specific validation, ensuring seamless integration of automated
testing into CI/CD pipelines while addressing unique challenges like model
accuracy, GenAI output validation, and ethical AI compliance.

Key
Responsibilities

Test
Automation Strategy & Framework Design

- Design and implement scalable
test automation frameworks for frontend (UI/UX), backend APIs, and AI/ML model-serving endpoints using tools like Selenium,
Playwright, Postman, or custom Python/Java solutions.

- Build GenAI-specific test suites for validating prompt outputs,
LLM-based chat interfaces, RAG systems, and vector search accuracy.

- Develop performance testing strategies for AI pipelines (e.g., model
inference latency, resource utilization).

Continuous
Testing & CI/CD Integration

- Establish and maintain continuous testing pipelines integrated with GitHub Actions,
Jenkins, or GitLab CI/CD.

- Implement shift-left testing by embedding automated checks
into development workflows (e.g., unit tests, contract testing).

AI/ML
Model Validation

- Collaborate with data
scientists to test AI/ML models for accuracy, fairness, stability, and bias mitigation using tools like TensorFlow
Model Analysis or MLflow.

- Validate model drift and retraining pipelines to
ensure consistent performance in production.

Quality
Metrics & Reporting

- Define and track KPIs.

- Test coverage (code, data, scenarios)

- Defect leakage rate

- Automation ROI (time saved vs. maintenance




effort)

- Model accuracy thresholds

- Report risks and quality trends
to stakeholders in sprint reviews.

- Drive adoption of AI-specific
testing tools (e.g., LangChain for LLM testing, Outstanding Expectations for
data validation).

Soft
Skills

- Strong problem-solving skills
for balancing speed and quality in fast-paced AI development.

- Ability to communicate
technical risks to non-technical stakeholders.

- Collaborative mindset to work
with cross-functional teams (data scientists, ML engineers, DevOps).

Requirements

Technical
Requirements

Must-Have

- 10 years in test automation,
with 2+ years validating AI/ML systems.

- Expertise in: Automation tools: Selenium, Playwright,
Cypress, REST Assured, Locust/JMeter

- CI/CD: Jenkins, GitHub Actions,
GitLab

- AI/ML testing: Model validation, drift
detection, GenAI output evaluation

- Languages: Python, Java, or JavaScript

- Certifications: ISTQB Advanced,
CAST, or equivalent.

- Experience with MLOps tools: MLflow, Kubeflow, TFX

- Familiarity with vector databases (Pinecone, Milvus) and RAG workflows.

- Strong programming/scripting
experience in JavaScript, Python, Java, or similar

- Experience with API testing, UI
testing, and automated pipelines

- Understanding of AI/ML model testing, output evaluation, and non-deterministic
behavior validation

- Experience with testing AI chatbots, LL​M responses, prompt engineering
outcomes, or AI fairness/bias

- Familiarity with MLOps pipelines and automated validation of model performance
in production

- Exposure
to Agile/Scrum methodology and tools like Azure Boards

📌 Software Testing Lead (Noida)
🏢 Leading
📍 Noida