1
Posted 3d ago•Remote
Subject-Matter Experts - AI Benchmark Question Writers & Reviewers
SeniorRemoteSalary undisclosed
Required Skills
Apache Spark
Job Description
1-StopAsia is building a native-language benchmark that tests how well today's leading AI models reason in Asian and European languages. We are looking for native-speaking academics and professionals to write expert-level exam questions that the best AI models cannot answer correctly, and to review questions written by others. Your work will show AI developers where their models still fall short in your language, culture and field.
Languages and domains we are hiring for
Language / Domains
- Thai - Law
- Filipino (Tagalog) - Literature (Poetry / Classics)
- Vietnamese - International Relations and Political Science; Philosophy (including non-Western); Psychology; Religion and Religious Studies
- Korean - Economics (Finance / Business / Trade); History; International Relations and Political Science; Law
- Standard Russian - Several humanities and social-science domains – tell us your field
You may apply for more than one domain if you have the expertise.
What you will do
As a Question Writer:
- Write original, exam-style questions in your native language in your domain.
- Write both short-form questions (multiple choice or one exact answer) and long-form questions (one connected reasoning task, graded against a rubric).
- Test each question on 5 AI models (GPT, Gemini, Claude Opus, Kimi, Muse Spark) using free accounts; w
Ready to apply? Optimize your CV for this specific jobAI customizes your experience bullets and increases chances to get hired.