Quick Note — Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
AI-103 › Unit 1: Plan and manage an Azure AI solution › Choose the appropriate Foundry services for generative AI and agents › Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
Quick Note — Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
A retail team routes 10 million short support messages a day into six fixed intent categories, and puts a frontier reasoning model behind it because "accuracy matters". The bill arrives: the model spent reasoning tokens deliberating over a six-way choice, ten million times, and reasoning tokens are billed as output tokens. A small language model answers the same question at a fraction of the cost, because the task was never hard — it was just frequent. The selection error was not picking a weak model; it was never asking what the task actually required.
Retrieval target
- Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
- Closed-book recall
- 5 minutes
- Open Microsoft Learn after a miss
Decision anchors
| Prompt | Compact answer |
|---|---|
| Order of elimination | Modality first — it rules out whole families and no capability argument overrides it. Then complexity and the cost/latency envelope, which picks the tier. Then placement — residency, egress, offline operation — which can eliminate every cloud-hosted candidate at once. Evaluate on your own data last, because measurement is the expensive step. |
| Small language model (SLM) | For narrow, high-volume, latency-sensitive work: classification, routing, extraction with a fixed label set. The Phi-4 family is the reference example. Reach for it when the task has a small answer space and volume dominates cost. |
| Frontier reasoning model | For multi-step reasoning, planning, and tool-calling orchestration. Generates reasoning tokens that never appear in the message content but occupy the context window and are billed as output tokens — so it is the most expensive way to answer an easy question. |
model-router | A deployable model that assesses each request and forwards it to a suitable model in its pool. Earns its place on mixed traffic (simple majority, hard minority). On a homogeneous stream of one simple task it adds a dispatch step and still routes to a small model. |
| Embedding model | Returns a dense vector, not an answer or a label. text-embedding-3-large supports similarity search and RAG grounding. If a stem asks for a category, an embedding alone is never the answer — a classifier still has to sit on top. |
| Foundry Tool vs Foundry Model | A Tool is a prebuilt capability with a maintained schema or classifier — Document Intelligence, Speech, Content Safety, Vision, AI Search. Choose a tool when the task is well-defined and the schema already exists; choose a model when the task is open-ended or nothing prebuilt covers it. |
| Placement: on-device | Data-egress rules, sovereignty, or genuinely offline sites are answered by running the model on the device (Foundry Local), not by picking a smaller cloud model. A small model in the cloud still sends the data off the device. |
Read the answers once, then cover the right-hand column and reconstruct each one from the prompt. A useful answer names the requirement, the capability that satisfies it, and the nearest alternative it rejects. If you can only recognize the answer after seeing it, retrieval is not yet secure.
Ninety-second explanation
Without notes, explain:
- Which property of a task makes a small language model the right choice rather than a cheap frontier model?
- Why are reasoning tokens a cost concern even when the visible answer is short?
- When does
model-routerstop being worth its dispatch step? - What kind of requirement overrides every capability argument?
- Give a task where a Foundry Tool beats a well-prompted frontier model, and say why.
Then check yourself against current Microsoft Learn. Record the missing decision rule, not merely the missed product name, in your error log.
Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools — quick retrieval
When to go deeper
Go deeper when a scenario turns on a specific limit, SKU capability, or preview status — those move, and a compact note cannot track them. Start from Foundry Models sold by Azure for the catalogue and availability, then the individual model's page for parameters.
Source and freshness
Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026). Reviewed 2026-08-20. Microsoft Learn controls changing product contracts — availability, limits, preview status, naming, and retirement dates move, and this note is deliberately compact.