LLM Engineering Expert – AI Evaluation Project | Codefeast

Location: (Remote)

Experience: 10+ Years

Job Type: Short-Term Contract

Start Date: Immediate

Domain: LLM Engineering | AI Evaluation | Engineering Simulation | Python | Coding Agents

Role Overview

We are looking for experienced LLM Engineering Experts with strong engineering design and simulation backgrounds to support an AI evaluation project. The role involves evaluating LLM-generated engineering solutions, assessing simulation outputs, identifying technical inaccuracies, and diagnosing reasoning and tool-use failures.

Key Responsibilities

- Evaluate AI-generated engineering solutions for technical accuracy and physical plausibility.
- Review and validate simulation results using domain-relevant engineering tools.
- Identify errors in engineering calculations, boundary conditions, unit consistency, and convergence criteria.
- Analyze LLM and coding agent outputs to identify reasoning and tool-use failures.
- Audit AI execution trajectories and diagnose failure patterns.




- Use Python scripting to support engineering simulations and technical evaluations.
- Document findings and provide detailed technical feedback to improve AI model performance.

Key Requirements

- Master's degree or PhD in Electrical, Mechanical, Aerospace Engineering, or a related discipline.
- 10+ years of hands-on engineering design experience.
- Proficiency in at least one open-source simulation tool, such as ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, or Gmsh.
- Strong Python scripting skills.
- Hands-on experience with modern LLMs and coding agents.
- Understanding of AI evaluation concepts, including pass@k, failure-mode analysis, and nondeterministic behaviour.
- Solid technical documentation and analytical skills.
- Excellent attention to detail and engineering precision.

📌 LLM - Engineering Expert (India)
🏢 Codefeast
📍 India