EazyML, Recognized by Gartner, EazyML (www.
EazyML.com) specializes in Responsible AI. Our solutions facilitate proactive compliance and sustainable automation and The company is associated with breakthrough startups like Amelia.ai. You will be responsible for testing and validating GenAI
- and Agentic AI-powered solutions for pharmaceutical Clinical Operations. This is a hybrid role combining Clinical Operations domain expertise with functional QA and software testing capabilities. You will help ensure that our AI solutions not only function correctly but also produce clinically accurate, meaningful, and reliable outputs. This includes understanding clinical trial workflows, reviewing requirements, developing and executing test cases, identifying defects, and validating AI-generated insights and recommendations. The ideal candidate has practical experience in Clinical Operations, understands software testing methodologies and basic QA automation, and is comfortable working with GenAI and Agentic AI technologies. You do not need to be an advanced programmer or AI engineer, but you should be able to work effectively with technical teams and evaluate solutions from both a functional and clinical operations perspective.
Key responsibilities
Core QA and test planning
• Review business requirements, user stories, workflows, and acceptance criteria to ensure solutions meet clinical operations needs.
• Develop and execute functional, integration, regression, exploratory, and user acceptance test cases.
• Validate application functionality, reports, dashboards, conversational interfaces, and AI-powered workflows.
• Identify, document, reproduce, and track defects through resolution using Jira or similar tools.
• Maintain test plans, test cases, test data, execution results, and defect reports.
• Collaborate with developers, product managers, and domain experts to resolve functional issues and improve solution quality. Pharma clinical operations domain and preferably GxP validation
• Apply working knowledge of clinical trial processes, including study start-up, site selection, patient recruitment, screening, enrollment, site performance, monitoring, and trial execution.
• Validate clinical operations workflows, business rules, operational metrics, and AI-generated insights against real-world clinical trial practices.
• Assess the accuracy and relevance of trial performance metrics, including enrollment rates, screening failures, site productivity, study milestones, and operational risks.
• Understand the role of clinical systems such as CTMS, EDC, eTMF, and related clinical trial platforms.
• Review AI-generated clinical operations reports, summaries, and recommendations for domain accuracy, completeness, and practical usefulness.
• Apply awareness of Valuable Clinical Practice (GCP), data integrity, confidentiality, and regulated pharmaceutical environments.
• Support risk-based validation and appropriate documentation for solutions used in GxP-relevant workflows, where applicable. GenAI and LLM evaluation
• Evaluate GenAI-generated responses for factual accuracy, relevance, completeness,
consistency, and alignment with supporting information.
• Identify hallucinations, unsupported claims, incorrect calculations, missing information, and misleading conclusions.
• Verify that AI-generated clinical insights and recommendations are grounded in appropriate source data and documents.
• Develop representative test scenarios, reference datasets, and expected outcomes for clinical operations use cases.
• Test AI responses across different prompts, clinical scenarios, input variations, and edge cases.
• Understand the non-deterministic nature of LLM outputs and apply appropriate evaluation criteria rather than relying solely on exact text matching.
• Collaborate with clinical domain experts and engineers to improve AI response quality and reliability. Agent and RAG testing
• Understand the fundamentals of Retrieval-Augmented Generation (RAG) and AI agents, including how they retrieve information, use tools, and execute multi-step workflows.
• Test AI-powered workflows from a business-user perspective to ensure tasks are completed accurately and logically.
• Validate information retrieved from clinical trial documents, operational databases, and other approved sources.
• Check whether AI agents correctly interpret user requests, follow business rules, and produce appropriate outputs.
• Test exception handling, missing or conflicting information, and situations requiring human review or escalation.
• Verify that changes to data sources, prompts, models, or workflows do not introduce functional or domain-related errors.
• Identify potential risks involving sensitive clinical data, inappropriate information disclosure, or unsupported automated actions. Automation and CI/CD
• Execute and maintain automated functional and regression test cases using established QA tools and frameworks.
• Use tools such as Selenium, Playwright, Postman, or similar platforms for application and API testing.
• Identify opportunities to automate repetitive test scenarios and improve testing efficiency.
• Apply basic scripting or programming knowledge, preferably Python, to support test execution and data validation.
• Understand how automated testing fits into software development and CI/CD processes.
• Collaborate with engineering teams to develop and improve automated test coverage.
• Explore the use of GenAI tools to assist with test case generation, test data preparation, test automation, and defect analysis. Quality ownership
• Ensure solutions meet functional requirements and clinical operations expectations before customer delivery.
• Define and apply practical acceptance criteria for functional and AI-generated outputs.
• Communicate defects, test outcomes,
quality risks, and release readiness to project stakeholders.
• Support user acceptance testing and customer validation activities.
• Maintain clear testing documentation and evidence of validation.
• Collaborate with domain experts and engineering teams to continuously improve solution reliability.
• Incorporate customer feedback and previously identified issues into regression testing.
• Help establish repeatable QA processes for GenAI
- and Agentic AI-powered applications. Required qualifications
• Bachelor's degree in Life Sciences, Pharmacy, Biotechnology, Clinical Research, Computer Science, Information Technology, or a related discipline.
• Approximately 4–8 years of relevant professional experience combining Clinical Operations, clinical systems, software QA, or functional validation.
• Strong working knowledge of pharmaceutical Clinical Operations, including the clinical trial lifecycle, operational processes, terminology, and performance metrics.
• Experience with functional software testing, test case development, defect tracking, regression testing, and user acceptance testing.
• Ability to interpret clinical operations requirements and translate them into realistic test scenarios.
• Working understanding of Generative AI, Large Language Models (LLMs), RAG, and Agentic AI concepts, including their capabilities and limitations.
• Familiarity with software test automation concepts and tools; advanced programming expertise is not required.
• Basic understanding of API testing, data validation, and scripting concepts.
• Strong analytical and problem-solving skills, with the ability to identify both functional defects and clinically incorrect or misleading outputs.
• Good written and verbal communication skills and the ability to collaborate with business and technical teams. Nice to have
• Experience working with pharmaceutical companies, CROs, clinical technology vendors, or life sciences consulting organizations.
• Hands-on experience with clinical operations platforms such as CTMS, EDC, eTMF, or clinical analytics applications.
• Experience testing or validating GenAI applications, conversational AI solutions, RAG systems, or AI agents.
• Familiarity with AI evaluation frameworks such as Ragas, DeepEval, LangSmith, or similar tools.
• Basic Python, SQL, or JavaScript scripting experience.
• Experience with QA automation tools such as Selenium, Playwright, Postman, or pytest.
• Exposure to CI/CD environments, Git, GitHub Actions, Azure DevOps, or Jenkins.
• Knowledge of GCP, GxP, GAMP 5, computer system validation, Computer Software Assurance (CSA), 21 CFR Part 11, and ALCOA+ data integrity principles.
• Experience validating clinical trial analytics, operational dashboards, KPIs, risk indicators, or AI-generated decision-support insights.
• Familiarity with data privacy, security, and responsible AI principles in pharmaceutical applications.
• Experience working in Agile development environments or with globally distributed technology teams.
📌 Pharma/Clinical Operations QA/Test Engineer (India)
🏢 EazyML
📍 India