What Actually Makes Something an Agent

What this slice covers

"Agent" is the most over-used word in current AI marketing, which makes it a genuinely hard exam topic: candidates often know how to build one before they can say what one is. This note is definitional. It covers what an agent is in Microsoft Foundry, the two kinds you will meet, what distinguishes an agent from a chat call with a good system message, and how an agent is identified — a subject that turns out to matter far more than it sounds.

The definition, unpacked

A prompt-based agent is a declaratively defined single agent that combines a model from the Foundry catalogue, instructions, tools, and natural language prompts to drive its behaviour. Four ideas are packed into that sentence, and each is worth pulling out.

Declaratively defined means you describe what the agent is rather than writing the loop that runs it. The definition is data — a model name, an instruction string, a set of attached tools — and the service supplies the machinery.

A model at the core means every agent has a model doing the reasoning and generating the responses. An agent does not replace the model; it wraps it.

Instructions are the standing behaviour, the same artefact as a system message, now attached to a durable object rather than passed on each call.

Tools are the part that makes the word "agent" earned rather than decorative: the capacity to retrieve knowledge or take actions instead of only emitting text.

Agent versus a well-configured chat call

The honest answer to "how is this different from calling a model with a system message?" is: three things.

First, persistence. An agent is a resource that exists in your project between calls, with a name, a configuration, and a history. A chat call is a transaction.

Second, reach. An agent can be given tools and knowledge sources, so it can consult a document index or call an API mid-answer. A bare model call cannot do anything except produce text.

Third, an operational surface. Because the agent is an object, it can be versioned, traced, evaluated, monitored, and published — none of which is available to an ad-hoc prompt living in your application code.

If a scenario needs none of those three, you do not need an agent, and reaching for one adds ceremony without benefit. That judgement is exactly what a fundamentals exam is testing.

The two types

Foundry distinguishes two kinds of agent, and the distinction is about where the agent's logic lives.

Prompt-based agents are the declarative kind described above. You create, edit, version, test, evaluate, monitor, and publish them from the agents playground in the Foundry portal, or equivalently from the CLI, SDK, or REST API. They are the natural starting point.

Hosted agents are containerised agents you build in code, using supported frameworks or your own custom code, which Foundry Agent Service then deploys and manages for you. You do not edit a hosted agent in the agent-building UI — its behaviour is in the container — but you can still invoke, evaluate, monitor, and publish it.

The mental model that helps: prompt-based agents are configured, hosted agents are programmed. Both run on the same service and share the same surrounding operational tooling.

Identity: name, version, and why it bites

An agent is referred to in code by its name together with a version. Two consequences follow immediately.

The name is permanent. Once you name an agent, you cannot change it. Choose accordingly.

Versions are immutable. Each saved version is fixed; changing anything means saving a new version. That is what makes it possible to direct requests at a specific version, compare two versions against identical inputs, and roll back when a change makes things worse. It also means unsaved edits are drafts — you can test them, but they vanish if you leave the portal, and history, monitoring, and full evaluations all require a saved version.

The operational pattern that follows is get-or-create: create the agent once, persist its identifier, and reuse it across sessions and processes rather than recreating it on every run. And treat a 404 as terminal. If a call for an identifier you previously created returns not-found, the agent was deleted; recreate it. Retrying the same identifier in a loop never succeeds and only generates error traffic.

Mistakes people make

Calling every model integration an agent is the first — the word implies tools and persistence, not just a chatbot. Assuming an agent is a different model is the second; it is a wrapper around one. Expecting to edit a hosted agent in the portal UI is the third. And treating agent creation as an idempotent startup step, so every restart leaves another orphaned duplicate behind, is the fourth — and the one that shows up as a mysteriously cluttered project six weeks in.