Performance & Security Engineer (AI) (Bengaluru)

Performance & Security Engineer (AI) (Bengaluru)

19 Nov
|
Mareana
|
Bengaluru

19 Nov

Mareana

Bengaluru

AI Test and NFR Engineer – AI, Performance & Security

Role Overview

We are seeking an AI Test and NFR Engineer with 5+ years of experience with strong hands-on expertise in AI/GenAI Testing, AI-Assisted Quality Engineering, Performance Testing, Security Testing, and Non-Functional Testing to contribute to the quality engineering of enterprise-grade AI and SaaS products.

The ideal candidate should have strong experience in testing AI-powered features (LLM-based assistants, agents, ML-driven analytics), using AI to accelerate test design, test data, automation, and defect analysis, and designing performance, load, stress, scalability, reliability, endurance, and security tests for web applications, APIs, databases, and distributed systems.

The candidate will work closely with Engineering, Product, DevOps, and QA teams to identify AI quality, performance, and security risks early, design effective test strategies, identify bottlenecks and vulnerabilities, and ensure production-ready quality.

Must-Have: AI Testing & AI-Assisted QA

AI-Assisted QA: Hands-on use of AI for test design, test data generation, performance/security analysis, defect investigation, risk identification, and AI-assisted QA workflows.

AI/GenAI Application Testing: Experience testing LLM-based features, agents, and ML-driven functionality for accuracy, consistency, safety, and regression, with defined evaluation criteria.

AI Security & Privacy: Awareness of prompt injection and data leakage risks, and protected use of AI tools with production-like data.

NFR for AI Systems: Performance, scalability, and reliability testing of AI-backed services.

Key Responsibilities

1) AI/GenAI Application Testing

- Design test strategies and suites for AI-powered features such as LLM-based assistants, agents, search/RAG, and ML-driven analytics.
- Validate AI output for accuracy, relevance, consistency, grounding, and hallucination, using rubrics and acceptance thresholds suited to non-deterministic behavior.
- Build and maintain evaluation datasets and golden sets, and run AI regression testing for model, prompt, and data changes (evaluation frameworks such as Promptfoo, Ragas, or DeepEval are an advantage).
- Test agent and tool-calling workflows, including integrations with APIs and databases, error handling, and fallback behavior.
- Test safety, guardrails, and AI-specific security risks such as bias, prompt injection, jailbreaks, and data leakage, aligned with the OWASP Top 10 for LLM Applications.
- Test latency, throughput, concurrency, rate limits, and timeouts of AI-backed services under load.

2) AI-Assisted Testing & AI-Driven Analysis

- AI-Assisted Test Design: Use AI tools (e.g., Copilot, Claude, ChatGPT) to generate test scenarios, test data, API and performance scripts, and SQL validation queries, while critically validating AI output for accuracy and coverage.
- AI-Driven Analysis: Apply AI to analyze logs, API responses, and performance metrics for faster defect triage and root-cause identification, with awareness of data privacy when using production-like data.
- AI-Assisted QA Workflows: Build reusable prompts and AI-assisted workflows for the QA team and integrate them into regular test cycles.
- AI Result Validation: Verify that AI-generated test results, analysis, and pass/fail verdicts are correct by cross-checking them against requirements, source data, and independent manual or automated checks, and by spot-checking for false passes, missed defects, and hallucinated findings before relying on them.

3) Performance Testing, Tools & Monitoring

- Design and execute performance test strategies, plans, scenarios, and realistic workload models based on business and technical requirements.
- Perform load, stress, spike,



endurance/soak, scalability, and volume testing for REST APIs, web applications, backend services, databases, and integrated systems.
- Define and validate performance benchmarks and acceptance criteria for response time, throughput, latency, concurrency, and resource utilization.
- Analyze system behavior under peak and sustained loads, identify degradation patterns and bottlenecks across application, API, database, infrastructure, and network layers, and perform capacity analysis as users, transactions, and data volumes grow.
- Strong hands-on experience with Apache JMeter or similar tools, including parameterized and data-driven tests; BlazeMeter or other cloud-based platforms is an advantage.
- Use monitoring and observability tools (Grafana, Prometheus, ELK, Splunk, or equivalent) to correlate test results with CPU, memory, database, network, JVM/application, and service-level metrics, and analyze logs to find the root cause of degradation.
- Work with Engineering and DevOps teams to investigate bottlenecks and validate performance improvements; experience in cloud and distributed environments is preferred.
- Perform performance regression testing for major releases and architectural changes, support production-readiness assessments, and prepare reports with observations, risks, and recommendations.

4) Security Testing

- Strong understanding of web application and API security concepts.
- Good understanding of the OWASP Top 10 and common application security vulnerabilities; awareness of the OWASP Top 10 for LLM Applications is an advantage.
- Hands-on experience with Burp Suite and/or OWASP ZAP.
- Perform security testing of web applications, REST APIs, and backend services.
- Test authentication and authorization mechanisms, including JWT, OAuth, session management, and role-based access control.
- Perform security testing for common vulnerabilities such as:
- Injection
- Broken access control
- Authentication and session-related vulnerabilities
- Security misconfiguration
- Sensitive data exposure
- API security vulnerabilities
- AI/LLM-specific vulnerabilities (e.g., prompt injection, sensitive data leakage)
- Perform positive and negative security testing for APIs and application workflows.
- Identify, document, prioritize, and track security vulnerabilities.
- Work closely with developers to understand security findings and validate remediation.
- Perform security regression testing after vulnerabilities are fixed.
- Support vulnerability assessment and security readiness activities before releases.
- Experience with SAST, DAST, dependency/security scanning, or vulnerability management tools is an advantage.

5) API, Data & Quality Engineering

- API & Backend Testing: Hands-on experience in REST API testing using Postman, Swagger, or similar tools, including functional, negative, boundary, concurrency, performance, security, and integration testing. Ability to analyze API behavior, logs, and downstream service/database interactions is preferred.
- Database & Data Testing: Good understanding of SQL and database testing, including data validation, reconciliation, integrity, consistency, and database performance analysis across application layers and backend services.
- Functional & Exploratory Testing: Good understanding of SDLC/STLC with experience in functional, integration,



regression, smoke, sanity, and exploratory testing; ability to identify functional gaps, edge cases, dependencies, and quality risks.
- Quality Engineering & Collaboration: Work closely with Engineering, Product, DevOps, Infrastructure, and QA teams to identify performance, security, scalability, and quality risks early.

6) Qualifications & Experience

- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.
- 5+ years of experience in Software Quality Assurance / Quality Engineering.
- Strong hands-on experience in AI-assisted testing and AI/GenAI application testing, including LLM-based features, agents, or ML-driven functionality.
- Hands-on use of AI tools (Copilot, Claude, ChatGPT, or equivalent) for test design, test data, scripts, and defect analysis, with critical validation of AI output.
- Strong hands-on experience in Performance Testing and Security Testing.
- Strong experience with JMeter or equivalent performance testing tools.
- Hands-on experience with Burp Suite, OWASP ZAP, or equivalent security testing tools.
- Good SQL and database testing knowledge.
- Experience testing web applications, APIs, databases, and integrated systems.
- Experience working in Agile/Scrum environments.
- Good experience with JIRA or similar defect management tools.
- Experience working closely with Engineering, DevOps, and Product teams.

7) Skills & Attributes

- Robust understanding of how LLMs and AI systems behave (prompts, context, non-determinism, hallucination) and how to test them.
- Ability to define evaluation criteria for AI output, with awareness of responsible AI (bias, safety, privacy, security).
- Strong analytical and problem-solving skills.
- Strong understanding of performance engineering concepts and application bottleneck analysis.
- Strong understanding of web and API security principles.
- Ability to analyze large volumes of performance metrics and identify meaningful trends.
- Ability to investigate complex performance and security issues across multiple application layers.
- Strong understanding of distributed systems, APIs, databases, and cloud environments.
- Good understanding of requirements, business workflows, and data models.
- Strong attention to detail and a quality-first mindset.
- Ability to independently design effective AI, performance, and security test scenarios.
- Valuable understanding of manufacturing processes is preferred.
- Functional understanding of ERP systems and data models is an advantage.
- Exposure to CSV, GxP, or other regulated environments is a plus.
- Experience with AI/ML applications or data-heavy platforms is an advantage.
- Good communication and collaboration skills.
- Ability to work effectively in a fast-paced product engineering environment.

8) What We Offer

- Competitive salary with performance-based bonuses.
- Comprehensive health and wellness benefits.
- Opportunity to test and shape AI-powered enterprise platforms and AI-assisted QA practices.
- Culture focused on innovation, learning, and technical excellence.
- Regular workshops, training programs, and tech talks.

About Mareana

Mareana builds cutting-edge enterprise digital AI products focused on Manufacturing, Supply Chain, and Sustainability. Our AI-driven platforms solve complex data challenges and deliver real-time insights, enabling businesses to accelerate digital transformation.

We have delivered more than $1B+ in business value to global clients, including leading firms in life sciences, consumer goods, chemicals, food & beverage, and manufacturing. We are recognized by Gartner as a “Cool Vendor in AI” and have been featured in Silicon Review and Manufacturing Insights.

📌 Performance & Security Engineer (AI) (Bengaluru)
🏢 Mareana
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: performance & security engineer (ai) (bengaluru) / bengaluru