An AI application can sound convincing and still make a costly mistake. It might answer from the wrong document, miss an important exception or report that it completed a task when the business system was never updated.
Before release, the useful question is specific: what evidence shows that this application is suitable for the work people will give it?
Begin with the business consequence
Consider an illustrative internal purchasing assistant. It can explain a purchasing policy, prepare a request and send it for approval. Those are three different responsibilities. An incomplete explanation creates confusion. An incorrect request creates rework. Sending a request without the right approval can cross a control boundary.
Write those responsibilities down before choosing evaluation metrics. Identify who uses the feature, which information they may see and which actions require confirmation. Give each important failure an owner and an agreed response.
Make a small set of examples worth trusting
Build a reviewed set of realistic questions and tasks. Include ordinary work, exceptions, incomplete requests and examples where the correct response is to ask for clarification. Keep the expected outcome and its business rationale with each case.
For the purchasing assistant, that could include an expired supplier record, conflicting policy versions, a request with a missing cost centre and a user who may prepare but cannot approve a purchase. These examples expose different risks. A collection of reworded happy-path questions will not provide the same information.
Separate source quality from answer quality
When an application uses retrieval-augmented generation, or RAG, inspect what it retrieves as well as what it says. Microsoft’s evaluation guidance treats retrieval relevance and groundedness as distinct concerns. A useful answer needs suitable information and must use that information appropriately. Microsoft’s RAG evaluation guidance.
Add an explicit test for missing or conflicting evidence. If a policy cannot be found, does the assistant say so? If a document has been superseded, can the answer still lead the user to the current rule? These are business acceptance questions that the team can decide together.
Check actions where they happen
For a tool-using agent, inspect the resulting state in the connected system. In the example, verify that the request contains the right values and reached the authorised approver. Also check that it was not submitted twice after a timeout.
Security scenarios should include hostile instructions in documents or tool results, inappropriate data access and attempts to exceed the permitted task. OWASP’s 2026 guidance covers both LLM application risks and the additional concerns of agentic systems. LLM Top 10 2026, Agentic Applications 2026.
Review the scoring as well as the product
If another model grades the answers, check its judgements against reviewed examples. Google’s documentation explicitly describes comparing model-based scores with human ratings. An automated judge needs evidence of suitability for the task too. Evaluating a judge model.
Repeat important variable tasks and investigate inconsistent outcomes. Record the configuration tested, material failures and unresolved questions. A release summary should explain what the team knows and what it still needs to control.
Start with a manageable set of meaningful cases, make the evidence reviewable and retain it for the next change. That gives the business a clearer basis for deciding whether to release, restrict the feature or keep improving it.
Planning an AI-enabled release? Talk to luvo about the testing questions your programme needs to answer.
