AI Testing Engineer, Amazon Bedrock

Location: 100% remote, seeking candidates located in LATAM.

Contract Length: 1 year, with potential for extension.

Contract Rate: $20-25 per hour in USD. Final rate will be determined based on relevant experience and overall fit for the position.

NTT DATA is seeking an AI Testing Engineer, Amazon Bedrock to define and execute the quality strategy for Generative AI applications and AI agents used in Customer Service and Contact Center scenarios. This role will evaluate AI outputs across deterministic and non-deterministic scenarios using automation, semantic and statistical analysis, human review, and adversarial testing while partnering with AI engineers and functional QA teams.

Responsibilities:

• Define the testing and evaluation strategy for Generative AI applications and AI agents.
• Create and maintain evaluation datasets, golden datasets, and regression suites covering representative business scenarios.
• Design single-turn and multi-turn conversational test cases, including context retention and workflow transitions.
• Evaluate AI outputs for accuracy, relevance, completeness, groundedness, faithfulness,