AI & Automation QA Engineer (Bengaluru)

AI & Automation QA Engineer (Bengaluru)

15 Nov
|
Es Magico AI Studio
|
Bengaluru

15 Nov

Es Magico AI Studio

Bengaluru

Job Description
AI & Automation QA Engineer

Experience:
3–5 Years | Employment Type: Full-Time |

Location:
Bengaluru / Mumbai / Hybrid About Es Magico Es Magico is an AI-first technology studio building intelligent, scalable technology solutions for startups, enterprises, and institutions. With a team of 70+ engineers and experience across 40+ clients, we specialise in AI engineering, fintech, SaaS platforms, and enterprise application development. We combine product thinking, engineering expertise, and rapid execution to take ideas from conceptualisation to production. Our culture emphasises AI-first development, continuous innovation, and building technology that delivers measurable business outcomes.

About the Role
We are looking for an AI & Automation QA Engineer who can ensure the quality, reliability, and performance of both traditional software applications and AI-powered products. The ideal candidate will have strong fundamentals in manual testing and automation engineering, along with an interest in testing up-to-date AI applications, LLMs, and autonomous AI workflows. You will work closely with developers, product managers, and engineering teams to identify issues, build comprehensive testing strategies, automate repetitive testing processes, and ensure production-ready software.

Key Responsibilities
1. Manual & Functional Testing
- Perform functional, regression, integration, system, exploratory, and end-to-end testing.
- Create and maintain test plans, test scenarios, and detailed test cases.
- Validate web applications, mobile applications, APIs, and backend workflows.
- Identify, document, track, and verify bugs across development cycles.
- Ensure application usability, compatibility, and consistency across browsers and devices. 2. Test Automation
- Design, develop, and maintain automated test suites for web and mobile applications.
- Work with automation frameworks such as Playwright, Selenium, Cypress, or Appium.
- Perform API automation using Postman, REST Assured, or equivalent tools.




- Integrate automated tests into CI/CD pipelines.
- Identify opportunities to improve test coverage and reduce repetitive manual testing. 3. AI & LLM Testing
- Test AI-powered applications, chatbots, voice agents, and LLM-based workflows.
- Evaluate AI-generated responses for accuracy, relevance, consistency, and hallucinations.
- Validate prompts, structured outputs, tool calling, and multi-step agent execution.
- Test RAG applications for retrieval accuracy, context relevance, and response grounding.
- Develop evaluation datasets and test scenarios covering edge cases and unexpected inputs.
- Perform regression testing across different models, prompts, and AI application versions. 4. Quality Engineering & Reporting
- Collaborate with developers to identify root causes and prevent recurring defects.
- Maintain test documentation, bug reports, test execution reports, and quality metrics.
- Validate application performance, reliability, and basic security scenarios.
- Participate in sprint planning, release testing, and production readiness assessments.
- Establish repeatable QA processes and continuously improve testing practices. Required Technical Skills
- 3–5 years of experience in software testing and quality assurance.
- Strong understanding of manual testing methodologies and SDLC/STLC.
- Hands-on experience with Playwright, Selenium, Cypress, or equivalent automation frameworks.
- Experience with API testing using Postman or similar tools.
- Understanding of JavaScript/TypeScript, Python, or another programming language.
- Working knowledge of SQL, databases, browser developer tools, and debugging.
- Experience with Git,



bug-tracking tools such as Jira, and Agile development practices.
- Familiarity with web and mobile application testing. Preferred Skills
- Experience testing LLM-based applications, AI chatbots, or voice AI products.
- Familiarity with prompt engineering, RAG, embeddings, and AI agent workflows.
- Exposure to AI evaluation frameworks such as DeepEval, Ragas, or promptfoo.
- Experience with performance testing tools such as JMeter or k6.
- Understanding of CI/CD pipelines and automated test execution.
- Experience testing SaaS platforms and enterprise applications. What We Are Looking For
- Strong analytical and problem-solving abilities with attention to detail.
- Someone who can think beyond predefined test cases and identify unexpected failure scenarios.
- Ability to independently own testing activities across multiple projects.
- A quality-first mindset with a strong sense of ownership.
- Willingness to experiment with emerging AI testing tools and methodologies.
- Good communication and collaboration skills. What Success Looks Like
- Consistently identifying defects before production deployment.
- Building reliable and maintainable automated test suites.
- Improving test coverage and reducing repetitive manual testing.
- Establishing robust evaluation processes for AI-powered applications.
- Ensuring reliable, consistent, and high-quality software releases. Why Join Es Magico?
- Work on traditional software products as well as cutting-edge AI applications.
- Gain hands-on exposure to LLMs, AI agents, voice AI, and intelligent automation.
- Work with modern engineering and testing tools.
- Collaborate with experienced product and engineering teams.
- Contribute to building quality standards for an AI-first engineering organisation. We are looking for someone who understands that quality is not just about finding bugs, but about building confidence in software—whether it is traditional code or AI-generated intelligence.

📌 AI & Automation QA Engineer (Bengaluru)
🏢 Es Magico AI Studio
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: ai & automation qa engineer (bengaluru) / bengaluru