Business Analyst - AI Response Evaluation Specialist
About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About the Role
We are looking for a detail-oriented Business Analyst – AI Evaluation Specialist to support a project focused on assessing and comparing chatbot responses. You will evaluate multiple AI-generated outputs against defined business requirements and quality criteria, analyze business processes, and deliver structured assessments that help improve the accuracy and usefulness of AI systems. The ideal candidate combines strong business analysis, data interpretation, requirements documentation, and critical evaluation skills with the ability to communicate clear, evidence-based conclusions.
Key Responsibilities
- Gather, analyze, and document business requirements, stakeholder expectations, and project objectives.
- Compare multiple chatbot responses and assess their accuracy, relevance, completeness, and alignment with defined evaluation criteria.
- Conduct structured evaluations and provide clear, evidence-based final assessments of AI-generated outputs.
- Translate business requirements into functional specifications, evaluation criteria, and act
