Senior QA & Evaluation Engineer

Job Title: Senior QA & Evaluation Engineer

Key Skills: Test Automation, .NET, API Testing, CI/CD, Automated Testing, LLM Evaluation, AI Testing, Test Frameworks

Experience: 4+ years of experience.

Location: Costa Rica, Peru, Colombia, Bolivia.

Mode: Remote.

We at Coforge are hiring Mid-Senior QA & Evaluation Engineer (N-5) with the following skill set.

Key Responsibilities

· Design and maintain automated test suites covering unit, integration, API, and end-to-end testing.

· Integrate automated testing into CI/CD pipelines and establish automated suites as release quality gates.

· Build evaluation frameworks for AI capabilities, including scored rubrics, LLM-as-a-judge approaches, and normalized pass thresholds.

· Build and maintain held-out evaluation datasets, measure AI behavior against ground truth, and report defect escape and evaluation results as part of delivery metrics.

Required Skills & Qualifications

· 4+ years of experience in Quality Engineering with a strong focus on test automation.</