Customize language model outputs for domain tasks, such as compliance summarization and domain extraction
AI-103 › Unit 4: Implement text analysis solutions › Apply language model text analysis › Customize language model outputs for domain tasks, such as compliance summarization and domain extraction
Customize language model outputs for domain tasks, such as compliance summarization and domain extraction
"Customize" spans four mechanisms with very different costs. Prompting and schemas change nothing about the model. Grounding changes what it sees. Adaptation from examples builds an index in minutes. Training produces a model artefact and needs labelled data. Picking the lightest mechanism that satisfies the requirement is the skill — and most domain tasks need less than teams assume.
Why This Matters
The ladder has a cheap bottom rung. A schema with well-written field descriptions solves a surprising share of domain extraction with no training data at all.
Trained features need labelled data. Custom NER, custom text classification, and CLU are "customizable", which means you supply training data. No data, no model.
Compliance work has a non-negotiable property. Where wording must be traceable, extractive summarization and grounding matter more than fluency.
Prerequisites
- The core/legacy split and which features are customizable.
- Content Understanding's
fieldSchemaand its three methods. - Confidence scores and grounding via
estimateFieldSourceAndConfidence. - Extractive against abstractive summarization.
Learning Objectives
By the end of this lesson you will be able to:
- Place a requirement on the customization ladder.
- Design a schema that encodes domain vocabulary.
- Choose a trained feature and know what it demands.
- Build compliance summarization with traceability.
- Use pro mode where the task spans documents.
Building Blocks
The customizable features. "If a feature is customizable, you can train an AI model using our tools to fit your data specifically. Otherwise a feature is preconfigured, meaning the AI models it uses can't be changed." Customizable: Custom NER (core), Custom text classification, CLU, Orchestration workflow, Question answering (all legacy). Preconfigured: prebuilt NER, key phrase extraction, sentiment, summarization, language detection, entity linking, text analytics for health.
Schema-level customization. A Content Understanding fieldSchema whose entries carry a name, type, and description, with a method: extract (documents only), classify (with an enum), or generate. Plus confidence scores 0–1 and grounding via estimateFieldSourceAndConfidence.
Adaptation. Adaptive custom translation uploads 5–10,000 prealigned source–target segment pairs (≤500 chars per pair) and "builds a custom bilingual index in minutes to adapt supported LLMs".
Analyzer improvement. Custom analyzers support labelled training examples, and the 2026-06-01-preview version "improves training behavior by distilling patterns into the built analyzer for improved privacy and efficiency". Content Understanding Studio "offers an experience optimized for analyzer performance improvement including improving custom analyzers using data labeling techniques".
Cross-document reasoning. Pro mode — "multi-step reasoning" over multiple inputs plus reference data attached at analyzer creation, for "identifying inconsistencies, drawing inferences, and making decisions". Documents only, and preview.
Summarization shapes for compliance. Extractive — "selecting key sentences… preserving their original positions". Abstractive — "new… sentences… not directly copied".
Four rungs of customization
| Attribute | ||||
|---|---|---|---|---|
| Changes | The instruction | What the model sees | An index over your examples | A model artefact |
| Needs | Nothing | Your content | 5–10,000 pairs | Labelled data |
| Time | Immediate | Ingestion | Minutes | A training cycle |
| Fits | Most domain extraction | Domain knowledge and facts | Terminology and style | Stable, high-volume categories |
Deep Dive
The ladder, and why the bottom rung is underrated
Teams reach for training because "our domain is specialised". Often the requirement is met a rung or two lower.
Schema and prompt. A field description is a specification, and it can carry domain vocabulary directly: "The governing clause reference, in the form 'Section N.N'. If several apply, list the controlling one first." That teaches the task without teaching the model anything. A classify field with an enum encodes a domain taxonomy exactly — and being a closed set, it returns values downstream code can switch on.
Grounding. Where the gap is knowledge rather than format — the model does not know your policies, products, or prior decisions — retrieval solves it. This is the common misdiagnosis: a model that formats correctly and gets facts wrong needs grounding, not training.
Adaptation. Adaptive custom translation is the clearest instance: 5–10,000 segment pairs, an index built in minutes, adapting an LLM to your terminology without a training cycle.
Training. Custom NER, custom text classification, and CLU produce models fitted to your data — and require labelled data to do it. Worth it when categories are stable, volume is high, and consistency matters more than flexibility.
The diagnostic question: is the model failing at format, at knowledge, at terminology, or at categorisation? Format is a schema problem, knowledge is grounding, terminology is adaptation, categorisation at scale is training.
Customizing for a domain task
Diagnose the failure
Format → schema. Knowledge → grounding. Terminology → adaptation. Categorisation → training.
Compliance summarization
Compliance work has requirements ordinary summarisation does not, and they change the design.
Traceability. A reviewer must find the source of every statement. Extractive summarization delivers this by construction — it works by "selecting key sentences from the document and preserving their original positions", so every sentence appears verbatim in the source. Abstractive produces "new… sentences… not directly copied", which is precisely what a traceability requirement forbids.
Grounding as evidence. Where a generative approach is used instead, estimateFieldSourceAndConfidence provides grounding — "the specific regions in the content where each value was extracted or generated" — so a generated statement still carries a pointer to its origin. That is the generative equivalent of extractive traceability, and it is why the setting matters more here than in ordinary extraction.
Fixed structure. Compliance summaries are usually consumed against a checklist, so a schema with named fields beats free prose: one field per required element, each with its own confidence score, so a missing or low-confidence element is visible rather than buried.
Review routing. Confidence scores from 0 to 1 let high-confidence elements pass and low-confidence ones reach a human — "minimizing the cost of human review" without abandoning it. In a compliance context that routing is the control, not an optimisation.
Escalation. Where the question spans documents — does this submission satisfy the policy, is this invoice consistent with the contract — that is pro mode: multi-step reasoning over multiple inputs plus reference data at analyzer creation, documents only and preview.
Domain extraction
Extraction customization has a clean decision path.
Standard entity types → prebuilt NER, preconfigured, no work.
Your own categories, with labelled text → Custom NER, a core capability that builds "custom AI models to extract custom entity categories… using unstructured text that you provide".
Your own categories, without labelled text → a fieldSchema. Since documents are the usual input, extract is available alongside classify and generate, and confidence and grounding come with it.
Clinical content → text analytics for health, core and purpose-built. Rebuilding it with Custom NER is wasted effort.
Documents needing labelled improvement → custom analyzers support labelled training examples, and Content Understanding Studio provides "data labeling techniques" for improving them — a middle path between pure schema and a fully trained NER model. In 2026-06-01-preview, training "distills patterns into the built analyzer for improved privacy and efficiency".
Keeping determinism where it belongs
One boundary carries over from the orchestration objective and applies sharply to compliance.
Where a rule is enumerable and must be applied identically every time — a threshold, a required clause list, a mandatory field set — that check belongs in code or a rules engine, not in a customized model. It is exact, auditable, and repeatable, and a low temperature is not determinism.
The productive division is the same as before: the model extracts and summarises, deterministic logic decides, and the model explains the decision. Customization improves the extraction and the explanation; it should not be asked to guarantee the decision.
Worked Examples
Example 1 — auditable regulatory summaries. Compliance summaries must let a reviewer locate every statement in the source.
Extractive summarization, which works by "selecting key sentences… preserving their original positions" — so every sentence is verbatim and locatable. If a generative approach is preferred for structure, enable estimateFieldSourceAndConfidence so each generated field carries grounding to its source region, and use a schema with one field per checklist element.
Example 2 — domain extraction, no labelled data. Clause references and obligations must be extracted from contracts; the categories are organisation-specific and nothing is labelled.
Custom NER is customizable and therefore needs labelled text. With none, use a fieldSchema — extract is available since these are documents, plus classify for taxonomies and generate for obligations — with confidence and grounding. If accuracy later needs lifting, custom analyzers accept labelled training examples through Studio's data-labeling techniques.
Example 3 — right format, wrong facts. A summariser produces perfectly structured output that misstates policy details, and the team plans to train a model.
A knowledge gap, not a format gap. Training on format examples teaches nothing about the facts. Ground the model — retrieval over the policy corpus, or reference data — and measure with retrieval and groundedness. Training would spend a cycle and leave the errors in place.
Visual Explanations
The ladder:
Compliance summarization design:
Common Mistakes
Jumping to training when a schema would do.
Training to fix a knowledge gap. That is grounding.
Choosing a customizable feature without labelled data.
Using abstractive summarization where wording must be traceable.
Omitting estimateFieldSourceAndConfidence on compliance output.
Rebuilding clinical extraction instead of using text analytics for health.
Expecting pro mode on non-document content. It is documents only and preview.
Asking a customized model to guarantee a deterministic rule.
Free prose where a checklist schema is needed.
Practice Exercises
- Name the four rungs and what each requires.
- Which Language features are customizable, and what does that imply?
- Why is extractive summarization the compliance default?
- Output is well-formatted but factually wrong. What is the fix, and what is not?
- When does a domain task justify pro mode?
▶Answers
- Schema and prompt — nothing required, changes the instruction. Grounding — your content, changes what the model sees. Adaptation — 5–10,000 segment pairs, builds an index in minutes. Training — labelled data, produces a model artefact. Choose the lowest rung that satisfies the requirement.
- Custom NER (core), plus custom text classification, CLU, orchestration workflow, and question answering (legacy). Customizable means you "train an AI model using our tools to fit your data" — so labelled data is required. Preconfigured features "can't be changed".
- Because it works by "selecting key sentences from the document and preserving their original positions", so every sentence in the summary appears verbatim in the source and can be located. Abstractive produces "new… sentences… not directly copied", which a traceability requirement forbids.
- It is a knowledge gap, so the fix is grounding — retrieval over the domain corpus, reference data, or a function call to the system of record — measured with retrieval and groundedness. Training on more format examples teaches nothing about the facts and leaves the errors in place.
- When the question spans multiple documents and requires judgement — "identifying inconsistencies, drawing inferences, and making decisions" — with stable material supplied as reference data at analyzer creation. It is documents only and preview, and a deterministic comparison belongs in code instead.
Summary & Concept Map
Customization is a ladder, and the exam rewards climbing only as far as needed. Schema and prompt encode domain vocabulary in field descriptions and enum values with no data at all. Grounding fixes knowledge gaps that training cannot touch — the misdiagnosis worth avoiding. Adaptation turns 5–10,000 segment pairs into a bilingual index in minutes. Training — Custom NER, custom text classification, CLU — produces a fitted model and demands labelled data. For compliance summarization, prefer extractive summaries, whose sentences are verbatim and positionally preserved, or a schema with one field per checklist element carrying grounding and confidence so review routes by score. Escalate to pro mode only for cross-document reasoning, remembering it is documents only and preview — and leave enumerable, must-be-identical rules to deterministic code.
Sources and freshness
Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.