1
Posted 3d ago•Remote

Subject-Matter Experts - AI Benchmark Question Writers & Reviewers

SeniorRemoteSalary undisclosed
Required Skills
Apache Spark
Job Description

1-StopAsia is building a native-language benchmark that tests how well today's leading AI models reason in Asian and European languages. We are looking for native-speaking academics and professionals to write expert-level exam questions that the best AI models cannot answer correctly, and to review questions written by others. Your work will show AI developers where their models still fall short in your language, culture and field.

Languages and domains we are hiring for

Language / Domains

  • Thai - Law
  • Filipino (Tagalog) - Literature (Poetry / Classics)
  • Vietnamese - International Relations and Political Science; Philosophy (including non-Western); Psychology; Religion and Religious Studies
  • Korean - Economics (Finance / Business / Trade); History; International Relations and Political Science; Law
  • Standard Russian - Several humanities and social-science domains – tell us your field

You may apply for more than one domain if you have the expertise.

What you will do

As a Question Writer:

  • Write original, exam-style questions in your native language in your domain.
  • Write both short-form questions (multiple choice or one exact answer) and long-form questions (one connected reasoning task, graded against a rubric).
  • Test each question on 5 AI models (GPT, Gemini, Claude Opus, Kimi, Muse Spark) using free accounts; w

Similar Openings in AI & Machine Learning

View all in category➔