12 Nov
|
Brillio
|
Bengaluru
Consultant n Primary Skills
• CI/CD Pipeline, React (React Context / Redux), Django / Flask / FastAPI, Docker, CSS/SCSS/LESS, PyTest, PostgreSQL / MongoDB / Redis, Kubernetes, REST API / GraphQL Job requirements
• Role: QA Engineer – GenAI Agents and GitLab Duo
• Design and execute functional, integration, regression, security, and performance tests for GenAI agents and multi-agent workflows.
• Validate application and infrastructure code interrogation results for accuracy, evidence, severity, false positives, and false negatives.
• Test GitLab Duo, AI Catalog agents, shared CI/CD templates, pipeline triggers, reports, and security gates.
• Develop automated Python-based test suites and evaluation datasets to assess agent reliability, consistency, security, and overall quality.
Key Responsibilities
1. AI Agent and Workflow Testing
• Create and execute test plans for GenAI agents and agentic workflows.
• Validate single-agent, multi-agent, and orchestrated workflows.
• Verify agent instructions, tool usage, task routing, handoffs, and termination conditions.
• Test agent behaviour for valid, invalid, incomplete, and ambiguous inputs.
• Validate error handling, retry mechanisms, timeouts, and fallback scenarios. 2. Application Code Interrogation Testing Validate the application code interrogation agent against:
• Security vulnerabilities and OWASP Top 10 risks.
• Insecure coding practices.
• Improper authentication and authorization.
• Sensitive-data exposure and secret leakage.
• Missing input validation and error handling.
• Resiliency issues such as missing timeouts,
retries, bulkheads, and circuit breakers.
• Incorrect logging, monitoring, and exception-handling practices. Verify that each finding contains accurate:
• File name and line number.
• Issue description and technical evidence.
• Severity and confidence.
• Business impact.
• Recommended remediation.
• Classification as confirmed or potential finding. 3. Infrastructure Code Interrogation Testing Test the infrastructure interrogation agent against Terraform, Kubernetes, Helm, YAML, and CI/CD configuration files. Validate detection of:
• Single points of failure.
• Inadequate high availability and scaling.
• Insecure storage and missing encryption.
• Excessive network access.
• Unrestricted communication between security zones.
• Missing identity and access controls.
• Inadequate logging, monitoring, and alerting.
• Insecure secrets management.
• Misconfigured cloud and container resources. 4. SDLC Agent Validation Support testing of AI agents used across the software development lifecycle, including:
• Requirement and story decomposition.
• UX/UI specification generation.
• OpenAPI contract generation and governance.
• Architecture validation.
• Backend and frontend implementation assistance.
• Dependency and open-source package curation.
• Unit, integration, contract, and end-to-end test generation.
• Security review.
• Infrastructure planning and implementation. 5. Triage and Finding Validation Validate the interrogation-triage agent for:
• Finding normalization.
• Duplicate detection and deduplication.
• Correct categorization.
• Severity and priority assignment.
• Ranking based on exploitability, exposure, and blast radius.
• Correlation of findings from application and infrastructure agents.
• Accurate consolidation of related findings.
• Avoidance of false positives and unsupported conclusions. 6. GitLab Dedicated CI/CD Integration Testing
• Validate agent execution through GitLab Dedicated pipelines.
• Test integration with shared CI/CD templates.
• Validate pipeline triggers for pushes, merge requests, and scheduled executions.
• Verify pipeline status, security gates, and failure behavior.
• Validate generated reports, artifacts, logs, and auditability.
• Test repository permissions and agent access controls.
• Verify behavior across different programming languages, repository structures, and infrastructure projects. 7. Test Automation
• Develop automated test cases using Python and suitable testing frameworks.
• Build reusable test datasets containing secure, vulnerable, and intentionally incorrect code.
• Automate regression testing for prompts, tools, agents, and n
📌 Lead Quality Engineer (Bengaluru)
🏢 Brillio
📍 Bengaluru