Role Overview
As a Test Automation Lead at Dailoqa, you’ll architect and implement robust testing frameworks for both software and AI/ML systems. You’ll bridge the gap between traditional QA and AI-specific validation, ensuring seamless integration of automated testing into CI/CD pipelines while addressing unique challenges like model accuracy, GenAI output validation, and ethical AI compliance.
Key Responsibilities
Test Automation Strategy & Framework Design
- Design and implement scalable test automation frameworks for frontend (UI/UX), backend APIs, and AI/ML model-serving endpoints using tools like Selenium, Playwright, Postman, or custom Python/Java solutions.
- Build GenAI-specific test suites for validating prompt outputs, LLM-based chat interfaces, RAG systems, and vector search accuracy.
- Develop performance testing strategies for AI pipelines (e.g., model inference latency, resource utilization).
Continuous Testing & CI/CD Integration
- Establish and maintain continuous testing pipelines integrated with GitHub Actions, Jenkins, or GitLab CI/CD.
- Implement shift-left testing by embedding automated checks into development workflows (e.g., unit tests, contract testing).
AI/ML Model Validation
- Collaborate with data scientists to test AI/ML models for accuracy, fairness, stability, and bias mitigation using tools like TensorFlow Model Analysis or MLflow.
- Validate model drift and retraining pipelines to ensure consistent performance in production.
Quality Metrics & Reporting
- Define and track KPIs.
- Test coverage (code, data, scenarios)
- Defect leakage rate
- Automation ROI (time saved vs. maintenance effort)
- Model accuracy thresholds
- Report risks and quality trends to stakeholders in sprint reviews.
- Drive adoption of AI-specific testing tools (e.g., LangChain for LLM testing, Excellent Expectations for data validation).
Soft Skills
- Strong problem-solving skills for balancing speed and quality in fast-paced AI development.
- Ability to communicate technical risks to non-technical stakeholders.
- Collaborative mindset to work with cross-functional teams (data scientists, ML engineers, DevOps).
Requirements
Technical Requirements
Must-Have
- 10 years in test automation, with 2+ years validating AI/ML systems.
- Expertise in: Automation tools: Selenium, Playwright, Cypress, REST Assured, Locust/JMeter
- CI/CD: Jenkins, GitHub Actions, GitLab
- AI/ML testing: Model validation, drift detection, GenAI output evaluation
- Languages: Python, Java, or JavaScript
- Certifications: ISTQB Advanced, CAST, or equivalent.
- Experience with MLOps tools: MLflow, Kubeflow, TFX
- Familiarity with vector databases (Pinecone, Milvus) and RAG workflows.
- Strong programming/scripting experience in JavaScript, Python, Java, or similar
- Experience with API testing, UI testing, and automated pipelines
- Understanding of AI/ML model testing, output evaluation, and non-deterministic behavior validation
- Experience with testing AI chatbots, LLM responses, prompt engineering outcomes, or AI fairness/bias
- Familiarity with MLOps pipelines and automated validation of model performance in production
- Exposure to Agile/Scrum methodology and tools like Azure Boards
📌 Software Testing Lead (Noida)
🏢 Dailoqa
📍 Noida