Microsoft describes AI evaluation as measuring the quality, safety and reliability of model or agent outputs. Evaluations can use prepared datasets, simulated conversations or records of actual runs. Scoring may use rules or another model. The important starting point is a defined task and criteria against which results can be assessed. Source: Microsoft Learn: AI evaluation and observability

Build a set of real questions

Choose representative enquiries about meal plans or cancellation terms. Record the evidence an acceptable answer should use. Include missing or conflicting information. This gives reviewers concrete cases to judge instead of relying on whether an answer sounds confident.

Repeat checks after changes

Run the same cases after changing the model, prompt or source material, then compare the answers and failures. Add newly discovered problems to the set. Keep human review for important decisions so a passing score is treated as evidence from those tests rather than a universal guarantee.

Written and updated by Travelgenix. Sources checked on 19 October 2026. Practical advice and examples are Travelgenix editorial guidance.