Evaluate models and apps, including detecting fabrications, relevance, quality, and safety
AI-103 › Unit 2: Implement generative AI and agentic solutions › Build generative applications by using Foundry › Evaluate models and apps, including detecting fabrications, relevance, quality, and safety
Evaluate models and apps, including detecting fabrications, relevance, quality, and safety
Evaluation appears twice in this certification for a reason. The responsible-AI objective covers the catalogue of evaluators; this one covers running an evaluation: choosing a model in the first place, building the dataset, generating adversarial cases you did not think of, and reading a result well enough to act on it.
Why This Matters
Fabrication has a specific measurement. "Hallucination" is not an evaluator. What you actually measure is groundedness — is the claim supported by the supplied context — plus ungrounded attributes for inferred properties. Naming the right instrument is the skill.
Model selection is an evaluation stage. Benchmarks and leaderboards shortlist; your own dataset decides. Skipping the first wastes effort, skipping the second ships someone else's benchmark as your quality bar.
You cannot only test cases you imagined. Adversarial simulators and the red teaming agent generate the inputs your dataset omits — which is the whole point of running them.
Prerequisites
- The evaluator families and what each requires as input.
- That groundedness measures support by supplied context, not truth.
- What a test dataset is: inputs, optionally context, optionally ground truth.
- Tracing into Application Insights.
Learning Objectives
By the end of this lesson you will be able to:
- Run the three-stage evaluation lifecycle and pick the right instrument at each.
- Detect fabrication with the correct evaluators, and know their input requirements.
- Build an evaluation dataset and decide whether ground truth is needed.
- Use simulators and the AI Red Teaming Agent to generate cases you did not author.
- Read a result — via cluster analysis and traces — and act on it.
Building Blocks
Stage one: model selection. The model catalog exposes benchmarks and leaderboards comparing models on standard tasks. Their job is to shortlist, not to decide — they measure general capability on public tasks, not your task on your data.
Stage two: pre-production. You supply a dataset and run evaluators against it. Three sources of test cases matter: hand-authored cases capturing known requirements; production-derived cases once traffic exists; and generated cases from adversarial simulators, which produce inputs designed to elicit failure. The AI Red Teaming Agent, built on PyRIT, automates adversarial probing across risk categories and attack strategies and returns a comparable scorecard.
Stage three: post-production. Operational metrics, continuous evaluation on sampled production traffic, scheduled evaluation against a fixed dataset — the mode that detects drift, because the inputs are held constant — scheduled red teaming, and Azure Monitor alerts on quality thresholds and harmful content.
The fabrication toolkit.
| Instrument | Measures | Needs |
|---|---|---|
| Groundedness | Support by supplied context, scored 1–5 | A judge model deployment |
| Groundedness Pro | Same, binary pass/fail with reasoning | No model deployment |
| Ungrounded attributes | Properties inferred and not supported | Context |
| Relevance | Does the response address the query | Query and response |
| Response Completeness | Does it cover what was asked | Query and response |
| Retrieval | Do retrieved chunks address the query | Retrieved context, no ground truth |
| Document Retrieval | Retrieval quality against labels | Ground truth labels |
Evaluation level. evaluation_level is "turn" (default) or "conversation", and levels cannot be mixed in one run.
Cluster analysis groups failing cases so a pattern is visible rather than a list of individual low scores.
Where test cases come from
| Attribute | |||
|---|---|---|---|
| Covers | Known requirements | Real usage | What you did not imagine |
| Available | Immediately | Only after traffic exists | Immediately |
| Bias | Your assumptions | Current users only | Adversarial by design |
| Tooling | Manual | Traces and sampling | Simulators; red teaming on PyRIT |
Deep Dive
Naming the fabrication problem precisely
A scenario rarely says "measure hallucination". It describes a symptom, and the symptom names the evaluator.
"The assistant states facts not present in the source documents" → groundedness. This is the core fabrication measure, and the choice between the two variants is decided by the deployment constraint: the AI-assisted evaluator scores 1–5 and requires a judge model, while Groundedness Pro returns binary pass/fail with reasoning and requires none.
"It attributes characteristics to people that the source never stated" → ungrounded attributes, which targets inferred properties specifically rather than unsupported statements generally.
"It answers a different question" → relevance. "It answers part of the question" → response completeness. Neither is fabrication; both are frequently offered as distractors alongside it.
And the framing to keep: groundedness is not truth. A response faithfully grounded in a stale or wrong document scores well. Content correctness is an ingestion and freshness problem, not something these evaluators measure.
Running an evaluation
Shortlist
Benchmarks and leaderboards in the model catalog — they narrow the field, they do not decide.
Datasets, and the ground-truth question
The first question about any dataset is whether reference answers exist, because it eliminates whole families.
With ground truth, the similarity evaluators (F1, BLEU, ROUGE, METEOR, GLEU, Similarity) and Document Retrieval become available. These are cheap, deterministic, and appropriate where a canonical answer genuinely exists — a lookup, an extraction, a classification.
Without ground truth, which is the common case for open-ended generation, you rely on reference-free evaluators: groundedness against supplied context, relevance and completeness against the query, coherence and fluency on the response alone, and the risk and safety set. This is not a lesser option — for a generative assistant there often is no single correct answer to compare against.
The mistake to avoid is manufacturing ground truth by having a model write the reference answers and then scoring against them. That measures agreement with the reference-writing model, not quality.
Simulators and red teaming: testing what you did not think of
A hand-built dataset has a structural blind spot: it contains the failures the author anticipated.
Adversarial simulators generate inputs designed to elicit unsafe or incorrect behaviour, giving coverage of cases nobody wrote down. The AI Red Teaming Agent systematises this — built on PyRIT, it scans across risk categories and applies attack strategies, returning a scorecard that is comparable between runs and across model versions. That comparability is what makes it useful as a release gate: the same scan before and after a change shows whether risk moved.
Because risk can reappear when a model version, system prompt, or tool set changes, red teaming also belongs on a schedule in production, not only at release.
What it does not do: replace content filters, or replace safety evaluation on ordinary traffic. It measures what the filters and prompt are letting through under provocation. An application can be adversarially robust and still produce harmful output on innocuous prompts — which is why the risk and safety evaluators apply to every generative application, not only ones in obviously sensitive domains.
Reading a result
A score tells you something changed; it does not tell you what.
Cluster analysis groups failing cases so a pattern emerges — fifty scattered low scores are noise, fifty that all involve date arithmetic are a defect with an address. Traces then supply the mechanism: which tool ran, what it returned, whether retrieval came back empty. Evaluator reasoning — the explanation attached to an AI-assisted score, or Groundedness Pro's pass/fail rationale — is often the fastest route to the cause.
Reading two evaluators together is especially diagnostic in RAG. Low retrieval with low groundedness means the content was never found. Good retrieval with low groundedness means it was found and not used — a prompting or context-window problem, not an indexing one.
Worked Examples
Example 1 — unsupported claims, no judge allowed. A policy assistant occasionally states requirements absent from the retrieved documents. Governance forbids deploying an additional model.
Groundedness Pro — it measures support by supplied context, returns binary pass/fail with reasoning, and requires no model deployment. The AI-assisted evaluator would give a 1–5 trend but needs a judge. Relevance measures whether the answer addresses the question, which is a different failure.
Example 2 — surprised after a model upgrade. A team selects a model from leaderboard rank, ships it, and finds domain quality worse than the previous model.
Benchmarks shortlist; they do not decide. The missing step is pre-production evaluation on their own dataset. Going forward, a scheduled evaluation against that fixed dataset makes the next version change measurable, because the inputs are held constant and any score movement is attributable.
Example 3 — coverage of what nobody wrote down. A team has 200 hand-authored test cases and wants confidence before release that unsafe outputs cannot be provoked.
Adversarial simulation plus the AI Red Teaming Agent on PyRIT, producing a scorecard across risk categories and attack strategies — repeatable, so before-and-after comparison is meaningful. Adding more hand-authored cases extends only the coverage the team already imagined.
Visual Explanations
The lifecycle and its instruments:
Naming the failure:
Common Mistakes
Looking for a "hallucination" evaluator. Fabrication is measured by groundedness and ungrounded attributes.
Reading groundedness as factual accuracy. It measures fidelity to the supplied context.
Choosing Groundedness when a judge deployment is excluded. Pro requires none.
Selecting a model on leaderboard rank alone. Benchmarks shortlist; your dataset decides.
Generating ground truth with a model and scoring against it. That measures agreement with the reference writer.
Relying only on hand-authored cases. Simulators and red teaming cover what you did not imagine.
Treating red teaming as a substitute for filters or safety evaluation on ordinary traffic.
Using continuous evaluation to detect drift. Drift needs a fixed dataset.
Mixing evaluation levels in one run. Not permitted.
Practice Exercises
- Which evaluators measure fabrication, and which measures inferred properties specifically?
- A judge deployment is forbidden. Which groundedness option, and what does it return?
- What does a benchmark leaderboard legitimately decide, and what must not be skipped after it?
- Why can a large hand-authored dataset still leave a release under-tested?
- Retrieval scores well, groundedness poorly. Where is the defect, and what would the reverse mean?
▶Answers
- Groundedness (and Groundedness Pro) measure whether claims are supported by the supplied context. Ungrounded attributes targets inferred properties that the source never stated. Relevance and Response Completeness are different failures and common distractors.
- Groundedness Pro — preview, powered by Azure AI Content Safety, returning binary pass/fail with reasoning and requiring no model deployment.
- It legitimately produces a shortlist of candidate models on public tasks. What must not be skipped is pre-production evaluation on your own dataset, since benchmark rank does not predict performance on your data.
- Because it contains only the failures the authors anticipated. Adversarial simulators and the AI Red Teaming Agent (PyRIT) generate cases outside that imagination and produce a comparable scorecard for before-and-after release gating.
- Good retrieval with poor groundedness means the content was found and not used — a prompting or context-window problem. The reverse means it was never found — ingestion, chunking, or query mode.
Summary & Concept Map
Evaluating an application is a three-stage practice. Benchmarks and leaderboards shortlist models and decide nothing. Pre-production runs your dataset — hand-authored, production-derived, and crucially simulated adversarial cases — through evaluators chosen by their required inputs, with red teaming on PyRIT producing a comparable scorecard as a release gate. Post-production splits into continuous evaluation on sampled traffic and scheduled evaluation on a fixed dataset, which is what detects drift, plus scheduled red teaming and alerts. Fabrication specifically is groundedness — 1–5 with a judge, or Pro, binary and judge-free — plus ungrounded attributes for inferred properties, remembering that groundedness measures fidelity to context and not truth. Then cluster analysis finds the failing class and traces supply the mechanism.
Sources and freshness
Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.