LLM Engineering Expert – AI Evaluation & Simulation

Location: Remote

Openings: 30

Engagement: Independent contractor, up to 24 weeks

Commitment: 40 hours per week, including at least 4 hours of overlap with PST and weekend on-call availability

Compensation

USD $500 per completed task. Each task is expected to require approximately 30–40 hours. The equivalent displayed rate is $12.50–$17/hour; actual compensation is task-based.

Role Overview

Vraify is hiring experienced engineering experts to create and validate challenging simulation-based engineering design problems used to evaluate advanced AI agents. The work spans electrical, mechanical, aerospace, control systems, systems engineering, and robotics.

Key Responsibilities

- Author original engineering design tasks with competing constraints, explicit optimization targets, validated reference solutions, and objective autograders.
- Build, run, and validate problem environments using open-source simulation tools and custom Python test benches.
- Evaluate coding-agent outputs and execution logs across repeated trials to identify systemic failure modes.
- Refine task difficulty using empirical model-performance data without introducing ambiguity or missing information.
- Collaborate with AI researchers, pod leads,



and domain experts to integrate rigorous benchmarks into the evaluation pipeline.

Requirements

- Master’s degree or PhD in Electrical Engineering, Mechanical Engineering, Aerospace Engineering, or a closely related field.
- 10+ years of hands-on engineering design experience.
- Proficiency with at least one relevant open-source simulation package, such as ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, or Gmsh.
- Strong Python scripting skills.
- Hands-on experience with modern LLMs or coding agents and evaluation concepts such as pass@k, failure-mode analysis, and nondeterministic behavior.
- Strong attention to physical plausibility, unit consistency, boundary conditions, convergence criteria, and technical documentation.
- Reliable high-speed internet and a personal desktop or laptop suitable for remote work.
- Availability for 40 hours per week, at least 4 hours of PST overlap, and weekend on-call coverage.

Eligible Locations:

Bangladesh, India, Indonesia, Pakistan, Vietnam, Egypt, Ghana, Kenya, Nigeria, Turkey, Brazil, and Colombia.

Mandatory Resume Requirement

Please ensure your resume includes your email address, current location, and phone number. This is mandatory.

📌 Artificial Intelligence Engineer (India)
🏢 Vraify
📍 India