Qualification: Skilled software engineer based in South Asia with experience building or reviewing realistic programming tasks.
Job Type: Independent contractor
Location: Remote
Domain: Software Engineering | AI Evaluation | Test Automation
Role Overview
We are conducting a paid study on the effectiveness of coding evaluation harnesses used to test AI agents. In this AI-moderated session, you will walk through a series of proposed coding tasks and environments, reviewing their structure, difficulty, and realism. You will provide technical feedback on the evaluation harnesses and suggest improvements to the test suites.
Your insights will help ensure these tasks accurately reflect real-world engineering requirements and that AI coding assistants are evaluated against high-quality development scenarios.
Key Responsibilities
- Review proposed coding tasks and programming environments for realism.
- Provide technical feedback on the structure and difficulty of evaluation harnesses.
- Walk us through your approach to verifying complex code challenges.
- Suggest improvements to make the test suites more robust.
Key Requirements
- Professional software engineer based in South Asia (backend developers, test automation engineers, and full-stack engineers welcome).
- Experience building or reviewing realistic programming tasks.
- Familiarity with coding evaluation harnesses or test automation.
Role Details
- Commitment: Fully remote, flexible schedule.
- Payment: Processed weekly based on services rendered.
- Terms: Engaged as an independent contractor; no access to confidential or proprietary information.
📌 Software Engineer – AI Code Evaluation | $40/hr (India)
🏢 Codefeast
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.