Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
AI-103 › Unit 1: Plan and manage an Azure AI solution › Choose the appropriate Foundry services for generative AI and agents › Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
Model selection is the first architectural decision in a Foundry solution and the one most often made backwards. Teams reach for the most capable model available, discover the bill, and then tune prompts to claw back cost that a different model would never have incurred. This lesson builds a repeatable selection method: eliminate cheaply, measure late, and let the requirement — not the leaderboard — name the model.
Why This Matters
A model choice sets your cost curve, your latency floor, and your failure modes for the life of the workload. Three properties make it unusually unforgiving.
Cost scales with volume, and reasoning is invisible. Reasoning models generate reasoning tokens in addition to the tokens you can see. Those tokens "never appear in the message content, but they occupy space in the context window and are billed as output tokens." A summarization endpoint returning 200 visible words can be billed for thousands of tokens. At ten million requests a day, choosing a frontier reasoning model for a six-way classification is not a small inefficiency — it is the entire budget.
Capability you do not use is capability you still pay for. A model that can plan a multi-step research task brings that machinery to every request, including the ones that need a single label. The extra depth does not make a fixed-label classification more correct; the answer space was six options either way.
Some constraints cannot be bought off. Data residency, egress prohibitions, and genuinely offline operation are absolute. No amount of model quality satisfies a rule that says the audio must not leave the device. These constraints eliminate whole deployment families in one move, which is why they belong early in the decision rather than late.
The exam tests this as a reading skill. A well-written scenario names the deciding property in one sentence — ten million messages a day, no connectivity at the site, values feed a pricing engine — and the rest is context. Learning to find that sentence is most of the objective.
Prerequisites
Before this lesson you should be comfortable with:
- The distinction between a model and a deployment — you deploy a model and address it by the deployment name you chose, which need not match the underlying model name.
- What an embedding is: a dense vector representing meaning, used for similarity search rather than for generating an answer.
- The basic shape of a Foundry project and the fact that models are reached through a project endpoint.
- That Foundry Tools are prebuilt capabilities — Document Intelligence, Speech, Language, Vision, Content Safety, AI Search — distinct from the generative models in the catalogue.
Learning Objectives
By the end of this lesson you will be able to:
- Apply a four-step elimination order — modality, complexity and envelope, placement, evaluation — to any selection scenario.
- Distinguish when a small language model outperforms a frontier model on the criteria that matter.
- Explain how reasoning tokens and
reasoning_effortchange the cost calculus of reasoning models. - Decide when
model-routerearns its dispatch step and when it is overhead. - Choose between a Foundry Tool and a generative model for a well-defined task.
Building Blocks
Frontier reasoning models. The most capable tier — multi-step reasoning, planning, and tool-calling orchestration. They deliberate internally, producing reasoning tokens that are billed as output. The reasoning_effort parameter governs how much they think, with supported values including none, minimal, low, medium, high, xhigh, and max. Crucially, effort is a request parameter, not a property of the deployment: one deployment can serve a none-effort routing call and a high-effort analysis call in the same pipeline.
Small language models (SLMs). Compact models sized for narrow, high-volume, latency-sensitive work — classification, routing, extraction against a fixed label set. The Phi-4 family is the reference example. They are not "worse" models; they are correctly sized ones.
Multimodal models. Accept images alongside text in a single request and reason over both — scenes, diagrams, charts. The right choice when the questions asked of an image are open-ended and unpredictable.
Embedding models. Return dense vectors. text-embedding-3-large is the reference. They enable similarity search and RAG grounding, and they never return a label or an answer on their own.
Image and video generation models. gpt-image-2 is generally available and supports arbitrary resolutions — both edges a multiple of 16 px, long edge up to 3,840 px (4K), aspect ratio up to 3:1. The gpt-image-1 series — gpt-image-1, gpt-image-1.5, gpt-image-1-mini — is in limited access preview and restricted to the fixed sizes 1024x1024, 1024x1536, and 1536x1024. Video generation is served by Sora 2 through an asynchronous jobs API.
Foundry Tools. Prebuilt capabilities with maintained schemas and classifiers. Document Intelligence returns extracted fields as strongly typed data — InvoiceDate as a date, SubTotal as a currency — so "normalization happens automatically without any configuration." Content Safety classifies harm categories and detects injection. AI Search performs hybrid retrieval with a semantic ranker.
model-router. A deployable model that assesses each incoming request and forwards it to a suitable model in its pool, behind a single endpoint.
Foundry Local. Runs a model on the device, for scenarios where connectivity is absent or data must not leave the hardware boundary.
What each family is for
| Attribute | ||
|---|---|---|
| Frontier reasoning | Multi-step reasoning, planning, orchestration | The answer space is small and fixed |
| Small language model | Narrow task, high volume, tight latency | The task genuinely needs deliberation |
| Multimodal | Open-ended questions about an image | The same fields are needed every time |
| Embedding | Similarity search, RAG grounding | You need a label or an answer |
| Foundry Tool | A maintained schema already covers it | The task is open-ended or novel |
Deep Dive
The elimination order
Selection is a filter, and the order matters because each step is cheaper than the next.
1. Modality. Absolute, and it eliminates whole families in one move. An embedding model cannot generate an image; an image model cannot classify text. No capability argument survives a modality mismatch, so this test costs nothing and removes the most candidates.
2. Complexity and the envelope. Now narrow within the surviving family. Ask what the task actually requires: does it need multi-step deliberation, or is it a single judgement against a small answer space? Then apply the envelope — throughput, latency target, cost per request. A narrow task at high volume with a tight latency target names a small language model without further argument.
3. Placement. Where is the model permitted to run? Data residency, egress prohibitions, sovereignty, and offline operation are constraints that can eliminate every cloud-hosted candidate regardless of tier. This step comes third because it is still free to apply and can invalidate everything above it — but it is written third because it is scenario-specific rather than universal.
4. Evaluate on your own data. Only now is there a short list worth measuring. Evaluation is the expensive step — it needs datasets, evaluator runs, and interpretation — so it goes last, against candidates that have already survived the free filters. Benchmarks comparing models on public datasets help at this stage; they do not replace measurement on your own task.
Eliminate cheaply, measure late
Modality
Absolute. Rules out entire families at zero cost.
Reasoning models and the cost of thinking
The economics of reasoning models are unintuitive because the expensive part is invisible. Reasoning tokens are billed as output tokens and consume the context window, and a single request "can spend anywhere from a few hundred to tens of thousands of reasoning tokens depending on how hard the problem is."
Two consequences follow. First, token analytics that count only visible output will understate the bill; read completion_tokens_details.reasoning_tokens on a Chat Completions response, or output_tokens_details.reasoning_tokens on a Responses API response, to see what was actually spent.
Second, the output cap covers reasoning too. max_completion_tokens (Chat Completions) and max_output_tokens (Responses) both cover "reasoning tokens, visible output tokens, and formatting tokens." A cap sized for the visible answer can therefore be exhausted before any visible output is produced — and the documented result is that "you pay for input and reasoning tokens but receive no answer." Applications must check status on every response rather than treating an empty body as an empty answer. The guidance suggests reserving around 25,000 tokens for reasoning plus output while learning a workload's appetite, then tuning down.
The lever that controls the spend is reasoning_effort. Because it is a request parameter, a pipeline can set it per step — none or low on mechanical routing and classification, higher only where deliberation earns its cost.
When routing helps, and when it does not
model-router is deployed like any other model and dispatches each request to a suitable model in its pool. It earns its place when request complexity genuinely varies — the classic shape being a large majority of simple lookups and a small minority needing depth. Cost then tracks complexity without the team building and maintaining a classifier.
It stops being worth the dispatch step when traffic is homogeneous. Ten million instances of the same simple classification have no complexity variance to exploit; the router adds a hop and still forwards to a small model. Choosing the small model directly is simpler and cheaper.
Routing also interacts with governance. Because the serving model becomes a per-request property, any requirement to state which model produced a given output means capturing that from the response and storing it with the output. Reconstructing it later fails, because the pool's contents and the router's behaviour change over time.
Tool or model?
The sharpest recurring decision is not which model but whether a model at all. Reach for a Foundry Tool when the task is well-defined, a schema or classifier already exists, and you want typed, consistent output with a maintained contract. Reach for a model when the task is open-ended, the output is prose, or nothing prebuilt covers it.
The trap is that a well-prompted frontier model will often produce the right values for a task a tool was built for. What it cannot supply is the guarantee: a maintained schema, automatic type normalization, and stability when the input layout changes. A prompt shifts that maintenance onto you, and the failure mode — a renamed field that parses cleanly and feeds a wrong number downstream — is silent.
Worked Examples
Example 1 — high-volume intent classification. Ten million short customer messages a day must be sorted into six fixed categories at the lowest cost per message.
Modality: text in, label out — every text model survives. Complexity and envelope: the answer space is six options and volume dominates cost, which names a small language model. Placement: unconstrained. Evaluate: measure candidate SLMs on a labelled sample of real messages. A frontier reasoning model would spend reasoning tokens on a six-way choice ten million times; model-router would add a dispatch step to homogeneous traffic and still route to a small model.
Example 2 — invoice field extraction feeding a pricing engine. Standard supplier invoices; the downstream service rejects anything not matching its schema.
The phrase standard plus typed values moves this out of model selection entirely. Document Intelligence with prebuilt-invoice returns fields whose schemas are "defined and maintained by Microsoft," with InvoiceDate as a date and SubTotal as a currency. A generative model would often produce the right values and would offer no guarantee of the right shape.
Example 3 — offline field inspection. Inspectors dictate findings at remote sites with no connectivity, and policy forbids audio leaving the device.
Placement decides this before anything else. Both constraints eliminate every cloud-hosted candidate regardless of capability or price, so the answer is an on-device deployment. Choosing a cheaper cloud model would satisfy neither constraint — the audio still leaves the device.
Visual Explanations
The selection filter, in order:
Where a Tool short-circuits the whole tree:
Common Mistakes
Choosing by capability instead of by requirement. The most capable model is the right answer only when the task needs that capability. A stem describing a narrow, high-volume task is not asking which model is best.
Treating reasoning models as having one cost per call. Effort is per request. A pipeline that leaves every step at the default effort pays frontier deliberation prices on its routing decisions.
Sizing the output cap for the visible answer. The cap covers reasoning tokens too, so a tight cap on a reasoning model produces billed requests with empty bodies.
Reaching for model-router on uniform traffic. Routing exploits variance. With none, it is a hop.
Assuming a prompt can replace a schema. A prompt asks; a tool guarantees. When the consumer is another system rather than a person, the guarantee is the requirement.
Applying placement constraints last. Discovering after benchmarking that no cloud model is permitted wastes the benchmark. Residency and egress are free to check and eliminate the most candidates.
Confusing GA with preview when a stem says "generally available". Among image models, only gpt-image-2 is GA; the gpt-image-1 series is limited access preview on fixed sizes.
Practice Exercises
- A workload classifies 8 million short messages daily into six categories, lowest cost per message. Which family, and which two properties decided it?
- A team migrates a prompt-tuned workload to a reasoning model and many requests return empty bodies while still being billed. Diagnose it and name the two parameters involved.
- Traffic is 90% simple FAQ lookups and 10% multi-step tax reasoning, on one interactive endpoint. What do you deploy, and what would provisioned throughput solve instead?
- Extraction of vendor, date, and total from standard invoices, output consumed by a pricing service. Tool or model, and what property makes it decisive?
- A regulated service routes through
model-routerand an auditor must name the model behind any archived output. What must the design do, and why does a later lookup fail?
▶Answers
- A small language model. Decided by the small fixed answer space (no deliberation required) and volume dominating cost.
- Reasoning tokens are generated before the visible answer and count against the same limit. Raise
max_completion_tokens(ormax_output_tokens) to leave headroom, and lowerreasoning_effortwhere depth is unnecessary. Confirm consumption incompletion_tokens_details.reasoning_tokens. model-router— complexity varies per request and the team wants no classifier. Provisioned throughput solves predictable latency and guaranteed rate limits, a different problem.- A Foundry Tool — Document Intelligence with a prebuilt model. Decisive property: strongly typed output against a maintained schema, because the consumer is a system that rejects anything off-shape.
- Capture the serving model from each response and store it with the output. A later lookup fails because the router's pool and behaviour change over time, so current state cannot reconstruct a past request.
Summary & Concept Map
Model selection is elimination, not comparison. Modality is absolute and free to check; complexity and the cost envelope pick the tier; placement can invalidate everything above it; evaluation on your own data comes last because it is the expensive step. Reasoning models are powerful and their cost is invisible — reasoning tokens are billed as output, share the output cap, and are governed per request by reasoning_effort. model-router converts complexity variance into cost savings and adds nothing to uniform traffic. And before choosing any model, ask whether a Foundry Tool already owns the task with a maintained schema.
Sources and freshness
Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.