About Us:Mobileum is a leading provider of Telecom analytics solutions for roaming, core network, security, risk management, domestic and international connectivity testing, and customer intelligence. More than 1,000 customers rely on its Active Intelligence platform, which provides advanced analytics solutions, allowing customers to connect deep network and operational intelligence with real-time actions that increase revenue, improve customer experience, and reduce costs. Know our story: https://www.Mobileum.Com/Headquartered in Silicon Valley, Mobileum has global offices in Australia, Dubai, Germany, Greece, India, Portugal, Singapore and UK with global HC of over 1800+.Join Mobileum TeamAt Mobileum we recognize that our team is the main reason for our success. What does work with us mean? Opportunities!Role: Senior QA Engineer – Gen AIAbout the RoleMobileum is looking for a highly skilled Senior QA Engineer – Gen AI to join our engineering organization in India. The ideal candidate will possess robust expertise in AI/ML testing, automation frameworks, Generative AI systems, and telecom technologies. This role will focus on validating LLM-based applications, AI-powered telecom platforms, and GenAI-driven solutions while ensuring reliability, safety, scalability, and compliance.We are looking for candidates with hands-on experience in testing GenAI and Agentic AI applications, including end-to-end agent workflows, Golden Dataset creation, fine-tuning/training dataset preparation, AI evaluation, and security testing of AI applications.Key ResponsibilitiesDesign and execute comprehensive test strategies for Generative AI and LLM-based applications.Validate LLM outputs for accuracy, relevance, coherence, groundedness, and safety.Design and execute test plans for single-agent and multi-agent systems,



including tool/function calling, planning, and multi-step task execution.Validate agent trajectories — verify the agent takes the right steps, calls the right tools with the right arguments, and recovers gracefully from errors, not just whether the final answer looks right.Validate guardrails, content filtering, and refusal behavior for out-of-scope or unsafe requests.Define and track quality metrics: accuracy, faithfulness, answer relevancy, context precision/recall, task success rate, tool-call accuracy, and consistencyTest prompts systematically across input variations, edge cases, and adversarial phrasings.Validate prompt robustness, consistency, and determinism (where expected) across runs and temperatures.Perform Bias Detection and A/B prompt testingBuild and maintain automation frameworks for UI, API, and AI workflow testing.Collaborate closely with AI engineers, product teams, DevOps, and telecom domain experts to improve overall system quality.Implement observability and monitoring validation using Prometheus, Grafana, and LLM tracing tools.Validate telecom AI use cases including Network anomaly detection, Fraud detection, Roaming analytics, Customer experience optimizationEnsure compliance with telecom and data privacy regulations including GDPR and DPDP.Experience with GenAI evaluation and observability tooling, e.G. DeepEval, RAGAS, Promptfoo, LangSmith, Langfuse, Arize Phoenix, TruLens, Braintrust,



OpenAI Evals.Experience with AI red-teaming / security tools, e.G. Garak, PyRIT, Giskard, or similar.Performance testing tools: Locust, k6, JMeter.Desired Profile4–7 years of experience in QA/Automation testing with exposure to AI/ML or GenAI systems.Strong programming skills in Python; knowledge of SQL and Java is preferred.Hands-on experience with AI/ML ecosystems and frameworks such as:LangChainLlamaIndexVector databases including Pinecone, Weaviate, and ChromaDBStrong experience with Playwright, Selenium, REST API, and gRPC testing.Familiarity with AWS, Azure, or GCP along with Docker and Kubernetes.Experience with observability and monitoring tools such as Prometheus and Grafana.Strong understanding of telecom business processes and telecom product ecosystems.Understanding of telecom-grade scalability, reliability, and high-availability systems.Knowledge of telecom regulatory and compliance requirements including GDPR and DPDP.Prior telecom product engineering/testing experience is highly preferred.Preferred ExperienceExperience testing enterprise-scale AI platforms.Exposure to Responsible AI and AI governance frameworks.Experience in performance and scalability testing for AI systems.Strong analytical, debugging, and communication skills.Hands-on knowledge of telecom protocols including SS7, Diameter, GTP, SIP, SCTPStrong understanding of mobile core network technologies including 4G/LTE, 5G SA/NSAExperience working on telecom use cases such as Roaming analytics, Fraud management, Revenue assurance, Customer experience analytics, Network intelligenceWork Experience: 4-7 YearsEducational Qualification:BE. / B.Tech., M.E./M. Tech. or M. Sc. in Computer Science or any related qualification.

📌 Senior Qa Engineer - Gen Ai (Bengaluru)
🏢 Mobileum
📍 Bengaluru