Define agent roles, goals, conversation-tracking approach, and tool schemas
AI-103 › Unit 2: Implement generative AI and agentic solutions › Build agents by using Foundry › Define agent roles, goals, conversation-tracking approach, and tool schemas
Define agent roles, goals, conversation-tracking approach, and tool schemas
An agent definition is four things: what it is (a role and a goal, expressed as instructions), what it can do (tool schemas), how it remembers (the conversation-tracking approach), and where it runs (prompt agent or hosted agent). Each is a decision with consequences, and the one people under-think is the tool schema — because the model chooses tools from the schema text alone.
Why This Matters
The model reads the schema, not your intent. Tool selection happens from names, descriptions, and parameter descriptions. A vague description is not a documentation problem; it is the direct cause of the wrong tool being called.
Prompt and hosted agents are a real fork. One is configuration only. The other runs your code on platform compute and gets its own Entra identity — which changes what you can build and how you secure it.
Conversation tracking is a decision, not a default. Threads carry context; hosted agents add session-level state persistence; anything that must be retained belongs in a store you own.
Prerequisites
- What function calling is: the model proposes a call, something executes it, the result returns.
- JSON Schema basics — types, required fields, enums.
- That agents are invoked through the Responses API.
- Managed identity as a principal that can hold role assignments.
Learning Objectives
By the end of this lesson you will be able to:
- Choose between a prompt agent and a hosted agent.
- Write instructions that express a role, a goal, and boundaries the model can act on.
- Design tool schemas that make correct selection likely.
- Choose a conversation-tracking approach and know its retention properties.
- Use agent versions as the record of what a definition was.
Building Blocks
The two agent kinds.
| Prompt agent | Hosted agent | |
|---|---|---|
| What you supply | Model, instructions, tools | Your code, packaged in a container |
| Compute | Platform | Platform compute running your container |
| Identity | The project's | Dedicated Entra identity per agent |
| State | Threads | Threads plus session-level state persistence |
| Fits | Most assistants | Custom runtime logic, private dependencies |
Instruction components. The documented prompt structure applies to agent instructions: Instructions, Primary content, Examples, Cue, and Supporting content. The techniques that matter here — start with clear instructions and repeat them at the end for recency, add clear syntax with separators or Markdown, give the model an out so it can say it does not know, and specify output structure.
Tool schemas. A function tool is declared with a name, a description, and a parameters JSON Schema whose properties carry their own descriptions. The model selects and populates the call from that text. Strict schema modes constrain the model to valid output.
Conversation tracking. A thread holds the exchange. Hosted agents add session-level state persistence. For durable retention under your control, bring your own Cosmos DB. Note that memory is in preview, so a design constrained to generally available capability cannot rely on it.
Agent versions. Configuration changes are automatically snapshotted, so instructions, model, and tool list at a point in time are recoverable.
Three places conversation state can live
| Attribute | |||
|---|---|---|---|
| Scope | One conversation | Hosted agent session | Whatever you write |
| Retention | Platform-managed | Platform-managed | Your policy |
| Fits | Ordinary multi-turn chat | Custom runtime needing durable session data | Compliance retention, e-discovery |
| Mechanism | Built in | Hosted agents only | Bring your own Cosmos DB |
Deep Dive
Instructions: role, goal, and the boundary
An agent's instructions do three jobs, and the third is the one that gets skipped.
Role establishes the persona and domain — who the agent is, whose questions it answers, what register it uses. Goal states what a successful turn looks like, which is what lets the model prefer one action over another when both are available.
Boundaries are the part that decides behaviour under pressure: what the agent must not do, what to do when the answer is not available, and when to escalate. Giving the model an out — explicit permission to say it does not know — measurably reduces fabrication, because a model with no sanctioned failure response will produce a confident one.
Two structural techniques carry over directly. Repeat the key instruction at the end, exploiting recency bias, since a long conversation pushes early instructions far from the generation point. And use clear syntax — separators, Markdown, or XML-style tags — so instructions, retrieved content, and user input are visually distinct. That distinction is not only for readability: it is what makes injected instructions inside retrieved content less likely to be read as system-level direction.
And the boundary that instructions cannot enforce: instructions are not a security control. Anything that must be impossible is enforced by not attaching the tool or by scoping the identity, never by asking the model nicely.
Defining an agent
Pick the kind
Configuration only → prompt agent. Custom code, private dependencies, or per-agent identity → hosted agent.
Tool schemas are the selection mechanism
This is the highest-value idea in the objective.
An agent picks a tool by reading the declared name, description, and parameter descriptions. There is no other channel. So a schema is not documentation attached to a capability — it is the capability as far as the model is concerned.
Three consequences follow.
Descriptions must say when, not just what. "Gets order data" tells the model nothing about choosing between it and three neighbouring tools. "Retrieves the full order record, including line items and shipping status, for a single known order ID. Use search_orders when the ID is unknown." names the boundary between them, which is the actual decision.
Overlapping tools cause wrong calls. If two tools could plausibly serve a request, the model will sometimes pick the other one. The fix is to make the schemas disjoint in their descriptions, or to merge the tools.
Parameters need constraints and descriptions. Types, enums, and required lists reduce malformed arguments, and each property's description tells the model what to put there. This is exactly what Tool Input Accuracy measures — an agent that selects correctly and passes a malformed date range is failing at the schema, not at reasoning.
Keep the count disciplined too: a large, overlapping tool list degrades selection. Curating the surface — the private tool catalog and a pinned toolbox version — is a quality measure as much as a governance one.
Conversation tracking, and what actually persists
Three mechanisms, three purposes.
Threads hold the conversation so the model sees prior turns. This is the default and covers ordinary multi-turn interaction. The constraint is the context window: a long thread eventually needs summarisation or trimming, and what gets dropped is a design decision rather than an accident you discover later.
Session-level state persistence is a hosted agent property — the custom runtime can keep durable state across a session rather than reconstructing it per turn.
Your own store is what a retention requirement means. Bring your own Cosmos DB puts thread and message data in your subscription under your retention and access policy. Traces are not the substitute: they are sampled, retention-bound, and structured for diagnosis.
Memory — the capability that carries facts across conversations — is in preview, which matters in any scenario restricted to generally available features.
Goals, and why they beat rules
A long list of prohibitions produces brittle behaviour: the model follows each rule and has no basis for choosing when they conflict or when the situation is unlisted.
A stated goal gives it that basis. "Resolve the customer's issue in as few turns as possible, escalating to a human when the account is flagged" lets the model reason about an unanticipated case in a way that twenty rules do not.
Rules still matter for genuine boundaries, and examples are the strongest way to convey format and tone — the documented prompt component. But structure the instructions as goal-first with boundaries attached, rather than as a rulebook the model must satisfy without knowing what it is for.
Worked Examples
Example 1 — the agent that calls the wrong lookup. Two tools exist: get_order and search_orders. The agent frequently calls get_order with a customer name instead of an ID.
A schema defect, not a reasoning failure. get_order must state that it takes a single known order ID and direct the model to search_orders when the ID is unknown; the order_id parameter needs a description and format constraint. Evaluation will surface this as Tool Selection and Tool Input Accuracy failures.
Example 2 — custom runtime with its own permissions. An agent must run proprietary scoring logic with private dependencies, and security requires its permissions be distinct from other agents in the project.
A hosted agent: your code in a container on platform compute, with a dedicated Entra identity per agent so role assignments are its own — and session-level state persistence for the runtime's working state. A prompt agent is configuration only and cannot host the logic.
Example 3 — seven-year conversation retention. Conversations must be retained for seven years under the firm's control and produced on request.
Bring your own Cosmos DB for conversation state, in the firm's subscription under its retention and access policy. Threads are platform-managed, and traces are sampled and telemetry-retained — neither is a system of record.
Visual Explanations
The four parts of an agent definition:
How a tool gets chosen:
Common Mistakes
Writing tool descriptions for developers. The model reads them at every selection.
Leaving two tools plausibly overlapping. Make the descriptions disjoint or merge them.
Omitting parameter descriptions and constraints. This is what Tool Input Accuracy catches.
Treating instructions as a security boundary. Remove the tool or scope the identity.
Writing a rulebook with no goal. Rules cannot resolve unanticipated cases.
Omitting an out. A model with no sanctioned failure response fabricates one.
Assuming a prompt agent can run custom code. That is a hosted agent.
Treating threads or traces as a retention mechanism. Bring your own store.
Relying on memory where only GA features are permitted. It is preview.
Practice Exercises
- Distinguish prompt and hosted agents on three axes.
- Why is a vague tool description a functional defect, and which evaluators expose it?
- Name three instruction techniques that reduce fabrication or drift, and say what each does.
- Where should conversation state live for a seven-year retention rule, and why not threads or traces?
- Why does a goal outperform a list of rules?
▶Answers
- What you supply — configuration only against your code in a container. Identity — the project's against a dedicated Entra identity per agent. State — threads against threads plus session-level state persistence. Hosted agents also run on platform compute.
- Because the model selects tools from the name, description, and parameter descriptions — that text is the only channel. A vague or overlapping description directly causes wrong calls, and shows up as Tool Selection and Tool Input Accuracy failures in evaluation.
- Give the model an out — explicit permission to say it does not know, which reduces fabrication. Repeat the key instruction at the end — recency bias, so it is not buried by a long conversation. Use clear syntax — separators or Markdown distinguishing instruction from retrieved content and user input.
- In your own store — bring your own Cosmos DB — under your retention and access policy. Threads are platform-managed, and traces are sampled and bound by a telemetry retention policy, structured for diagnosis rather than as a record.
- Because rules cannot resolve conflicts or unlisted situations, while a stated goal gives the model a basis for choosing. Boundaries and examples still matter, but goal-first instructions generalise where a rulebook is brittle.
Summary & Concept Map
Defining an agent is four aligned decisions. The kind forks between a prompt agent — configuration only — and a hosted agent, which runs your container on platform compute with a dedicated Entra identity and session-level state persistence. Instructions carry role, goal, and boundaries, using the documented techniques: give the model an out, repeat the key instruction at the end, and use clear syntax to separate instruction from content — while remembering instructions are never a security control. Tool schemas are the selection mechanism: names, descriptions, and parameter descriptions are read on every decision, so they must say when to use a tool, stay disjoint from neighbours, and constrain arguments. And conversation tracking runs from threads through hosted-agent session state to your own Cosmos DB where retention is required — with memory still in preview.
Sources and freshness
Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.