Research Engineer, Benchmarks
About the Role
Join a small, technical team working on AI evaluation and reinforcement learning infrastructure. You will design and build rigorous benchmarks for assessing AI agents on realistic, domain-specific workflows, helping research and engineering teams understand how well agents perform in practice.
What You'll Do
Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.
Work with subject-matter experts to translate real workflows into practical tasks and evaluation criteria.
Build reliable infrastructure to run models and agents against benchmark tasks at scale.
Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
Validate benchmark results against real-world evaluations and user needs.
Write clear technical documentation and reports that explain methods and results.
What We're Looking For
Two to four years of experience in software engineering, machine learning engineering, or research, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments.
