From Playground to Production: The Agent Development Lifecycle

The shape of the lifecycle

Getting an agent to answer one question well takes minutes. Getting one you would put in front of customers takes a process, and Microsoft Foundry models that process explicitly. The lifecycle runs from first creation through to production monitoring, and at the centre of every agent sits a model from the Foundry model catalogue that drives reasoning and response generation.

The steps, in order: choose an agent type; create the agent and start testing, iterating in the playground or in code; add tools and data; save meaningful milestones as versions; debug with tracing; evaluate quality and safety; optionally optimise a hosted agent's instructions automatically; publish and integrate; then monitor and iterate.

Read that list as a loop rather than a line. The last step feeds the second, and most real work is spent circling between testing, versioning and evaluation.

Two kinds of agent

Prompt-based agents are declaratively defined single agents combining a Foundry model, instructions, tools and natural-language prompts. You can edit, version, test, evaluate, monitor and publish them from the Agents playground in the Foundry portal, or build them through the CLI, SDK or REST API.

Hosted agents are containerised agents you build in code using supported frameworks or custom code, which Foundry Agent Service then deploys and manages. The distinction that matters is what you give up and what you keep: you do not edit hosted agents in the agent-building UI, but you can still invoke, evaluate, monitor and publish them. The management surface stays; the visual authoring surface does not.

Versions are the unit of progress

Versioning is not bookkeeping here — it gates real functionality. You can experiment freely with unsaved changes in the playground, but if you want conversation history, performance monitoring or full evaluations, you must save.

Four properties define the model. Versions are immutable once saved, so any change means creating a new version rather than overwriting an old one. Draft state lets you test unsaved changes, with the catch that leaving the portal loses them. Version control operations let you direct requests at a specific version, which is what makes controlled rollout and rollback possible. Version history lets you compare versions three ways: their configuration settings, their chat output given identical inputs, and their YAML definitions.

One irreversible decision hides in here. After you name an agent you cannot change the name, and in code you refer to the agent as <agent_name>:<version>. Choose the name deliberately.

Referencing agents without breaking things

Because the identifier is the contract between your application and the agent, treat it as durable. Create each agent once and reuse the identifier across sessions and processes rather than recreating it on every run. If a call returns 404 Not Found for an identifier that previously worked, the agent was deleted — that is terminal, not transient. Recreate it instead of retrying; repeatedly polling a deleted identifier never succeeds and only generates error traffic. The recommended pattern is get-or-create: at startup, look the agent up by name and create it only if the lookup returns 404, which keeps one stable identifier and avoids orphaned duplicates.

Tools, tracing and evaluation

Tools are how an agent gains knowledge — specific files or indexes — or the ability to act by calling external APIs, including custom Model Context Protocol server connections. There is a hard rule: to save an agent with a tool attached, the tool must be configured successfully first. If it needs authentication or a connection, that setup comes before saving, not after. Configured tools can be reused across agents.

Tracing is the debugging instrument for everything that follows. It lets you confirm whether the agent called the tools you expected, inspect tool inputs and outputs, and find latency hotspots across model and tool calls. This matters because agent misbehaviour is usually a wrong tool call rather than a wrong sentence, and only a trace shows you that.

Evaluation is the regression net. Run repeatable evaluations before publishing and after any meaningful change, so quality is measured consistently across versions rather than judged by whichever answer you happened to try.

Publishing, identity and monitoring

Publishing turns a version you are happy with into an agent application with a stable endpoint you can open in a browser, share, or embed in existing applications. You can update and republish at any time.

The security consequence is the single most missable point in this topic: permissions assigned to the project identity do not automatically carry over to the published agent. After publishing you must reassign the privileges the agent application's identity needs, and confirm it has only the access it needs — no more.

The surrounding practice follows normal engineering hygiene. Treat agent configuration like application code. Prefer least privilege and role assignments over embedded keys. Keep secrets in a managed secret store referenced through connections rather than hardcoded in code, configuration or prompts. After publishing, monitor quality and safety signals, review traces when behaviour changes, and republish when you fix something.

Common pitfalls

Four recur often enough to be worth memorising: unsaved changes are temporary; tools must be configured before an agent can be saved; publishing can require permission updates; and a deleted agent's identifier is invalid forever, so a 404 means recreate rather than retry.