Company: Evalixa AI Private Limited

Engagement Type: Independent Freelance Contributor

Work Mode: Remote

Location: India

Schedule: Flexible and task-based

Compensation: Per accepted task; rates will be communicated before assignment To apply: Email your details to [email protected].

About Evalixa AI Evalixa AI Private Limited develops high-quality datasets, technical evaluation tasks, and testing frameworks for advanced AI systems. Our work assesses whether AI systems can understand software repositories, investigate engineering problems, implement code changes, and complete tasks in realistic development environments.

About the Opportunity We are seeking Freelance Expert Contributors to review and validate repository-based software-engineering tasks used to evaluate advanced coding AI systems.

Tasks may be based on real open-source pull requests, bug fixes, feature implementations, refactoring, test improvements, and repository maintenance.

Contributors may receive a task package containing instructions, repository information, environment configuration, a reference solution, automated tests, execution logs, and metadata. The contributor must determine whether the task is accurate, reproducible, properly tested, and suitable for AI evaluation.

Work will be assigned according to project availability, technical expertise, assessment performance, submission quality, and business requirements.

Key Responsibilities

- Review software-engineering tasks based on repositories, issues, commits, and pull requests.
- Validate task instructions, code changes, reference solutions, tests, configurations, and logs.
- Confirm that tasks are technically correct, reproducible, properly tested, and licence-compliant.
- Identify missing requirements, weak tests, solution leakage, shortcuts, and environment issues.
- Classify tasks as valid, invalid, or requiring revision, supported by clear technical evidence.
- Complete required revisions and follow all quality, confidentiality, submission, and deadline requirements.

Typical Task Process

- Access the assigned task package.
- Read the instructions and acceptance criteria.
- Inspect the repository and related issue, commit, or pull request.
- Review the environment and dependency configuration.
- Examine the reference solution and code changes.
- Inspect the automated tests and evaluation logic.
- Review build, test, and execution logs.
- Run permitted validation commands when required.
- Identify inconsistencies, missing requirements, or test weaknesses.
- Submit a validity decision with clear technical reasoning.
- Complete revisions when requested.

Required Qualifications

- Strong background in software engineering, computer science, or a related field.
- Experience reading, debugging, testing, or maintaining software.
- Proficiency in Python, JavaScript, TypeScript, Java, Go, C++, Rust, or another programming language.
- Working knowledge of Git, repositories, commits, branches, patches, and pull requests.
- Ability to understand unfamiliar codebases and trace behaviour across files.
- Experience running test suites and investigating build failures.
- Familiarity with unit, integration, regression, or end-to-end testing.
- Ability to determine whether tests enforce written requirements.
- Understanding of dependencies, package managers, and build tools.
- Ability to follow detailed technical documentation.




- Strong analytical, problem-solving, and written communication skills.
- Ability to work independently and meet deadlines.
- Reliable computer and internet connection.

Preferred Qualifications

- Bachelor’s, Master’s, or Ph.D. in computer science, software engineering, AI, machine learning, data science, or a related field.
- Equivalent professional, research, or open-source experience will also be considered.
- Experience reviewing pull requests or contributing to open-source repositories.
- Familiarity with Linux, shell commands, Docker, containers, CI/CD, or sandboxed environments.
- Experience with GitHub Actions or similar continuous-integration systems.
- Experience developing or validating coding assessments, benchmarks, hidden tests, or automated grading systems.
- Experience with debugging, feature development, refactoring, or test engineering.
- Familiarity with coding assistants, language models, software-engineering agents, or AI evaluation.
- Understanding of solution leakage, benchmark contamination, licensing, privacy, and evaluation integrity.
- Relevant GitHub contributions, portfolios, publications, or work samples.

Quality Requirements Submissions must be technically accurate, complete, evidence-based, independently verifiable, and consistent with the instructions, repository, solution, and tests. All work must be submitted within the communicated deadline and comply with confidentiality, licensing, security, and data-handling requirements.

A submitted task is not automatically accepted. It may be returned for revision or rejected if it does not meet the required standards.

Compensation and Task Allocation

- Payment is calculated per task accepted after quality review unless different written terms apply.
- The applicable rate will be communicated before assignment.
- Rates may vary according to complexity, technical domain, and expected completion time.
- Tasks returned for revision may require corrections before acceptance.
- Rejected, incomplete, duplicate, late, or non-compliant submissions may not qualify for payment.
- No minimum number of tasks, working hours, earnings, or continuous assignments is guaranteed.
- Task allocation may depend on expertise, availability, accuracy, performance, and project demand.

Engagement Benefits

- Flexible and remote task-based work.
- Exposure to advanced software-engineering AI evaluation.
- Experience reviewing realistic repository-level engineering problems.
- Opportunities to work with different languages, frameworks, and technical domains.
- Access to relevant documentation and training.
- Opportunities to qualify for advanced assignments or reviewer responsibilities.

Selection Process

- Application review: Resume, education, experience, GitHub profile, portfolio, and work samples are reviewed.
- Initial screening: Shortlisted candidates attend a discussion about their technical background and availability.
- Technical assessment: Candidates complete a limited code-review,



debugging, test-analysis, or task-validation exercise.
- Quality evaluation: The assessment is reviewed for correctness, reasoning, attention to detail, and compliance.
- Onboarding: Successful candidates complete identity, contractual, confidentiality, compliance, and payment documentation.
- Training: Contributors complete the required documentation and qualification exercises.

Completing the selection or training process does not guarantee assignments, minimum earnings, or continuous participation. Application Requirements Applicants should submit:

- Updated resume or CV.
- LinkedIn profile.
- GitHub profile, portfolio, publications, or relevant work samples, where available.
- Details of programming languages, frameworks, and technical experience.
- Educational and professional information.
- Current availability.
- Computer specifications when requested.

Applicants must not submit confidential code, credentials, private repository content, or work samples they are not authorised to share. Freelance Relationship Selected contributors will work as independent freelancers and not as permanent employees of Evalixa AI Private Limited.

This engagement does not guarantee fixed hours, minimum tasks, minimum earnings, internships, placements, permanent employment, or employee benefits.

Contributors are responsible for their equipment, internet access, work schedule, taxes, and obligations applicable to independent professionals.

Confidentiality and Work Integrity Contributors may receive access to non-public repositories, tasks, solutions, tests, results, or documentation. Contributors must not:

- Publish, share, sell, or distribute assigned materials.
- Upload confidential content to public or unauthorised platforms.
- Disclose reference solutions, hidden tests, evaluation criteria, or results.
- Submit another person’s work or involve unauthorised third parties.
- Use AI or external tools when prohibited by project instructions.
- Submit fabricated evidence or misleading results.
- Retain project materials after deletion or return is requested.

Performance and Ending the Engagement Continued task allocation depends on technical accuracy, submission quality, responsiveness, reliability, communication, and compliance. Evalixa AI may pause assignments, request retraining, or end the engagement for poor-quality work, repeated missed deadlines, inactivity, failure to complete revisions, fraud, security violations, confidentiality breaches, or failure to follow project procedures.

Either party may end the engagement according to the applicable written agreement.

Applicant Privacy Evalixa AI may process applicant information for recruitment, assessment, identity verification, contracting, payment administration, security, fraud prevention, auditing, and legal compliance.

Information will be accessible only to authorised personnel and relevant service providers where necessary. Applicants may request access to, correction of, or deletion of their information, subject to applicable legal and retention requirements.

Equal Chance Evalixa AI Private Limited follows fair and merit-based selection practices. Selection and continued task allocation depend on technical capability, assessment performance, submission quality, reliability, project availability, and business requirements.

📌 Expert Contributor - SWE-bench task reviewer (Hyderabad)
🏢 Evalixa AI
📍 Hyderabad