Founding Engineer, Environments + Data
About the Role
Build evaluation environments and benchmarks that measure whether AI agents can perform realistic company work and meet or exceed human performance. This founding engineer role sits at the intersection of agent evaluation and production readiness, with direct influence on how agent competence is assessed.
What You'll Do
Design synthetic company environments with realistic email, records, and files for agents to use over extended periods.
Build robust graders and evaluation systems that can assess agents, including when their performance may exceed human reviewers' expertise.
Develop benchmarks that reflect real-world production performance.
Find and address evaluation failures, including reward hacking and benchmark gaming.
Simulate software tools such as inboxes and CRMs to support realistic agent interactions at scale.
What We're Looking For
Experience building environments or benchmarks used by frontier AI labs to train or evaluate agents.
Hands-on experience designing and implementing AI agent graders or evaluation systems.
Experience
