Founding Engineer, Environments and Data
About the Role
Build evaluation environments and benchmarks that measure whether AI agents can perform real company work and match or exceed human performance. This founding engineering role focuses on agent capability assessment and production readiness, helping establish reliable evidence of agent competence.
What You'll Do
Design synthetic company environments with realistic emails, records, and files generated from scratch.
Build graders and evaluation systems that can assess agents even when their performance may exceed human reviewers' expertise.
Develop benchmarks whose rankings correlate with real-world production performance.
Find and address evaluation failure modes, including reward hacking and benchmark gaming.
Simulate tools such as inboxes and CRMs with enough fidelity to support agent interactions at scale.
What We're Looking For
Hands-on research experience with synthetic data pipelines, benchmarks, evaluations, or post-training.
At least 1 to 2 years of experience at a fast-growing or AI-focused startup.
Experience building environments
