- Role: Software Engineer – Ruby (Remote)
- Location: Remote (Work from Anywhere)

Role Overview:

We are hiring for one of our clients, seeking a Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation) to work on a Full-Time basis. The projects focus on building LLM evaluation and training datasets that address realistic software engineering problems. One approach involves creating verifiable software engineering tasks from public repository histories using a synthetic method with human‑in‑the‑loop, while expanding coverage across programming languages and difficulty levels.

Key Responsibilities:

• Analyze and triage GitHub issues across trending open‑source libraries to identify suitable evaluation scenarios.

• Set up and configure code repositories, including Docker containerization and environment preparation for reproducible testing.

• Evaluate unit test coverage and quality metrics to assess baseline code health and test suite robustness.

• Modify and run Ruby codebases locally to measure LLM performance in bug‑fixing and code‑completion tasks.

• Collaborate with AI researchers to design and select repositories and issues that present challenging cases for LLMs.

Required Skills & Qualifications:





• Minimum three years of skilled software engineering experience with strong proficiency in Ruby and Ruby on Rails.

• Demonstrated expertise with Git, Docker, and basic software pipeline setup for reproducible testing.

• Ability to navigate and understand complex, real‑world codebases, and to run, modify, and test them locally.

• Experience with Ruby testing frameworks (RSpec, Minitest), gem management (Bundler), and code quality tooling.

• Prior contribution to or evaluation of open‑source projects is considered a plus.

More About the Opportunity:

This role provides a unique opportunity to work with a global leader in the artificial intelligence industry, contributing to how large language models interact with real code. The work directly influences the development of AI‑assisted software development tools and benchmarks.

Equal Opportunity Employer:

We hire based on skills and expertise. All qualified candidates are welcome regardless of background, experience, or prior employment history. Applications are reviewed solely on demonstrated technical ability and qualifications.

Apply Now!

📌 Software Engineer – Ruby (Remote) (Salem)
🏢 Quik Hire Staffing
📍 Salem