About The Role
This is a hands-on development role as the guardian of platform quality, not a manual testing role. Your work will involve curating test sets, coming up with extensive testing scenarios and enhancing our testing platform to incorporate these. You'll also make sure our services hold up under real traffic, and you'll build and look after the test data that all of this depends on.
What you'll do
• Curate and maintain test datasets: Curate and maintain sharply defined datasets for both ML and non-ML APIs, especially tricky inputs that we encounter in the real world.
• Scenario generation: Come up with all possible scenarios for testing over 200 APIs and their combinations, as well as internal services
• Automate everything: Your primary mandate is to test everything via automation, primarily through our in-house framework and tools. You will be actively writing code to enhance this framework. You should be able to leverage AI Native Engineering to the fullest extent.
• Test ML APIs: We build and ship a wide range of ML models which require extensive evaluation. You will be responsible for maintaining both the evals and the datasets.
• Run load and performance tests: You are responsible for a high throughput platform. You will be designing and executing performance tests to ensure the platform meets all SLOs
• Work closely with the engineering team: Reproduce and document bugs clearly,
help developers triage them, and suggest ways to make our systems easier to test. What we're looking for
• 2-4 years of experience in QA automation engineering or software development.
• Strong coding skills in at least one language. We care more about how well you code than which language you use: readable, well-structured, maintainable test code.
• Continuous learner. You are someone who stays updated, learns new skills and is excited about learning new technologies / languages.
• Practical experience with at least one load testing tool, such as Locust, k6, JMeter or Gatling, and the ability to interpret results with clarity.
• Some experience curating datasets, and an eye for the edge cases that break things.
• A basic understanding of how testing ML systems differs: outputs can vary, "correct" often means "above a threshold", and quality is measured across a dataset rather than one request.
• Comfort with Git, CI pipelines and the Linux command line.
• Clear written and verbal communication. Positive, timely bug reporting is critical. Nice to have (but not mandatory)
• Prior experience working with Python in a production setting. What we don’t want
• A “manual” QA engineer who’s looking to learn / switch.
• Someone who prefers their comfort zone
• People with remote / hybrid expectations
📌 Software Engineer Test (Mumbai)
🏢 IDfy
📍 Mumbai
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.