AI/ML Engineer — Data, Evaluation & Model Improvement
Focus: LLM evaluation, fine-tuning and continuous product improvement
Company: SecNinjaz Technologies LLP
Experience: Around 2 years of relevant hands-on experience
Employment: Full-time, on-site
Locations: Netaji Subhash Place, New Delhi,
About the opportunity
At SecNinjaz, we are building an AI-powered Vulnerability Assessment and Penetration Testing (VAPT) product that helps organisations identify, investigate and validate security weaknesses in authorised environments.
We are looking for an AI/ML Engineer who can build trustworthy datasets, evaluate model and agent performance, and turn measured weaknesses into practical improvements through better data, retrieval and fine-tuning.
You will own the engineering work that helps us answer three questions: Where does the system fail? What should we improve? Does the change perform better on unseen cases?
Robust applied AI/ML skills are essential. Cybersecurity or VAPT experience is a plus. Our security specialists will help define domain labels,



review evidence and validate security findings.

What you will work on
• Data pipelines: Transform approved agent traces, tool outputs, expert feedback and verified outcomes into structured, versioned datasets.
• Data quality: Implement cleaning, deduplication, annotation workflows, provenance tracking and sensitive-data handling.
• Evaluation datasets: Create training, validation and held-out test sets, preventing related cases or near-duplicate examples from leaking across splits.
• Model and agent evaluation: Build repeatable evaluations for task completion, tool-use accuracy, evidence quality, false positives, latency and compute cost.
• Failure analysis: Investigate whether failures originate from the model, prompts, retrieval, tools, data or workflow design, and recommend the appropriate improvement.
• Fine-tuning: Run reproducible supervised fine-tuning experiments, including approaches such as LoRA or QLoRA where s

📌 AI/ML Engineer — Data, Evaluation (India)
🏢 SecNinjaz Technologies
📍 India