AI Response Quality Evaluator (Gemini)

About Gramian

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

About the Role

We are seeking AI Personalization Evaluators to support Gemini Personalization Evals by reviewing AI-generated responses using personalized prompts and context retrieved from connected Google applications. You will assess response quality, contextual accuracy, and personalization, identifying subtle errors, unsupported assumptions, and opportunities to improve the user experience.

This role is suited to detail-oriented individuals with strong analytical thinking, excellent written communication, and active experience using Google apps such as Gmail, Google Calendar, Google Photos, and Google Drive.

CONTRACT: Contractor

LOCATION: Remote — Australia only

COMMITMENT: Minimum 4 hours per day, up to 40 hours per week

TIME OVERLAP: 4 hours of overlap with PST

DURATION: Up to 16 weeks

EVALUATION PROCESS: Shortlisted candidates will receive a Job Interest Form. Selected candidates will be contacted regarding consent and onboarding requirements.