LLM Engineering Expert – AI Evaluation Project | Codefeast

Location:
(Remote)

Experience:
10+ Years

Job Type
Short-Term Contract Start Date: Immediate Domain: LLM Engineering | AI Evaluation | Engineering Simulation | Python | Coding Agents Role Overview We are looking for experienced LLM Engineering Experts with strong engineering design and simulation backgrounds to support an AI evaluation project. The role involves evaluating LLM-generated engineering solutions, assessing simulation outputs, identifying technical inaccuracies, and diagnosing reasoning and tool-use failures.

Key Responsibilities
• Evaluate AI-generated engineering solutions for technical accuracy and physical plausibility.
• Review and validate simulation results using domain-relevant engineering tools.
• Identify errors in engineering calculations, boundary conditions, unit consistency, and convergence criteria.
• Analyze LLM and coding agent outputs to identify reasoning and tool-use failures.
• Audit AI execution trajectories and diagnose failure patterns.




• Use Python scripting to support engineering simulations and technical evaluations.
• Document findings and provide detailed technical feedback to improve AI model performance. Key Requirements
• Master's degree or PhD in Electrical, Mechanical, Aerospace Engineering, or a related discipline.
• 10+ years of hands-on engineering design experience.
• Proficiency in at least one open-source simulation tool, such as ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, or Gmsh.
• Strong Python scripting skills.
• Hands-on experience with modern LLMs and coding agents.
• Understanding of AI evaluation concepts, including pass@k, failure-mode analysis, and nondeterministic behaviour.
• Solid technical documentation and analytical skills.
• Excellent attention to detail and engineering precision.

📌 LLM - Engineering Expert (India)
🏢 Codefeast
📍 India