Job Position: Senior AI Quality Assurance (QA) Engineer
Experience: 7+ Years
Location: Bengaluru
Required Skills
7+ years of experience in Software Quality Assurance, Test Automation, or AI Quality Engineering.
Hands-on experience evaluating LLM-powered applications, conversational AI systems, NL2SQL solutions, or similar AI workflows.
Strong understanding of AI benchmarking methodologies and experience creating benchmark (golden) datasets for regression testing.
Experience with AI evaluation and observability platforms such as Lang Smith, MLflow, Arize Phoenix , or equivalent.
Solid understanding of AI evaluation metrics, including Precision, Recall, , , F1 Score, Exact Match (EM), Mean Reciprocal Rank (MRR), Execution Accuracy, latency (P50/P95/P99), token usage,



and cost analysis .
Solid proficiency in Python with hands-on experience building automation using Pytest , including unit testing, integration testing, API testing, exception handling, mocking, fixtures, and parameterized testing.
Experience testing REST APIs and distributed backend services.
Robust SQL skills with the ability to validate generated queries, execution results, schema alignment, and business logic.
Strong analytical and debugging skills with the ability to perform root cause analysis across complex AI systems.

📌 Senior Ai Quality Assurance Qa Engineer Bengaluru
🏢 DATAMAXIS
📍 Bengaluru