Design Azure infrastructure for AI apps and agent-based solutions

Infrastructure design for a Foundry solution is mostly about boundaries: which scope owns administration, which identity acts, where traffic is allowed to flow, and whose subscription holds the data. Get those right and the rest is configuration. Get them wrong and the remedies are migrations.

Why This Matters

Three properties make these decisions unusually sticky.

Scope decides blast radius. A role assigned at the wrong level grants far more than intended, and RBAC inherits downward. The difference between assigning at the Foundry resource and at a single agent is the difference between "can call every agent here" and "can call this one".

Some choices are made once. The resource type you provision determines which endpoints exist at all — and no later setting adds an endpoint the resource does not have. Networking and customer-managed keys are similarly awkward to retrofit onto a running estate.

Identity is the security design. RBAC only bites on the Microsoft Entra path: "if you use key-based authentication instead, the key grants full access without role restrictions." A careful role model plus enabled key auth is a role model with a bypass.

The exam presents this as a governance scenario — several teams, a central IT function, a regulator — and asks you to place scopes, roles, identities, and network boundaries correctly.

Prerequisites

• Azure RBAC basics: a principal gets a role at a scope, and assignments inherit downward.
• What a managed identity is and why it removes stored secrets.
• The distinction between the control plane (create, configure, delete) and the data plane (inference, agent calls).
• That a Foundry resource is reached through a project endpoint.

Learning Objectives

By the end of this lesson you will be able to:

1. Place the three Foundry scopes — resource, project, agent — and explain what each governs.
2. Select the correct built-in role for a persona, and know which roles not to use.
3. Design network isolation for prompt agents and hosted agents.
4. Apply bring your own resources for storage, search, and conversation state.
5. Explain why the resource type decision constrains the whole build.

Building Blocks

The three scopes. "Foundry resource: The top-level scope that defines the administrative, security, and monitoring boundary for a Microsoft Foundry environment." "Foundry project: A sub-scope within a Foundry resource used to organize work and enforce access control for Foundry APIs, tools, and developer workflows." "Agent: A narrower scope within a Foundry project that applies to an individual agent" — and note that agent-scope assignments "are currently evaluated only for agent endpoint access", not for broader control-plane permissions.

The built-in roles. Foundry Agent Consumer — "access to interact with agent endpoints… least-privilege access role for principals that only need to interact with agents." Foundry User — reader access plus data actions, the "least-privilege access role for developers building and testing agents." Foundry Project Manager — management actions on projects, build and develop, publish agents, and conditionally assign Foundry User. Foundry Account Owner — create projects and accounts, manage models, assign roles — but not build in projects. Foundry Owner — full management and build, described as "highly privileged".

The rename. These were previously Azure AI User / Owner / Account Owner / Project Manager. "The role IDs and core permissions are unchanged by the rename", and code should "use the role definition ID (GUID) instead of the role name" — Foundry User being 53ca6127-db72-4b80-b1b0-d745d6d5456d.

Roles not to use. "Don't assign built-in roles that start with Cognitive Services. These roles are designed for accessing AI Services resources directly and don't apply to Foundry scenarios. Similarly, don't use the Azure AI Developer role for Foundry work. Despite the name, this role is scoped to Azure Machine Learning workspaces and Foundry hubs, not to Foundry projects or Foundry hosted agents."

Networking. "Run agents within your Azure virtual network for full network isolation and compliance with data residency requirements. Private networking is available for prompt agents. Hosted agents support bring-your-own Azure Virtual Network (BYO VNet), where each session runs in a VM-isolated sandbox connected to your VNet."

Bring your own resources. "Use your own Azure resources (for example, storage, Azure AI Search, and Azure Cosmos DB for conversation state) to meet compliance and operational needs."

Agent identity. "Each agent can have a dedicated Microsoft Entra identity, enabling secure, scoped access to resources and APIs without sharing credentials" — automatic and per-agent for hosted agents.

Deep Dive

Placing scopes and roles

Start from the personas the scenario names, not from the role list.

A platform or central IT function that owns the estate but does not build maps to Foundry Account Owner — it can create accounts and projects and manage models, and explicitly cannot build in projects. That "cannot" is a feature: it keeps administration and development separable.

A team lead who creates projects for their team, builds in them, and publishes agents maps to Foundry Project Manager. Publishing is the discriminator — "To publish agents, you need the Foundry Project Manager role (minimum) on the Foundry resource scope."

A developer who builds and tests maps to Foundry User, the documented least-privilege role for exactly that.

A calling application or service principal that only invokes an agent endpoint maps to Foundry Agent Consumer — "If a user or service principal only needs to interact with agents (for example, calling the Responses API) without creating or modifying them, assign Foundry Agent Consumer instead of Foundry User."

The generic Azure roles are a trap here. On the permissions table, Contributor and Reader both show no for "build and develop in a project", so a developer given Contributor can create resources and still cannot do their job.

Narrowing to a single agent

Agent scope is the finest control and is often missed. Rather than moving an agent into its own project to isolate access, assign at the agent's own resource URI:

/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<account>/projects/<project>/agents/<agent>

"Any role that can be assigned at the project scope can also be assigned at the agent scope." The important caveat is that these assignments are "currently evaluated only for agent endpoint access" and "don't grant broader control-plane or management permissions" — so use them to narrow who may call an agent, not as a general administrative boundary.

Networking by agent type

Both agent types can be isolated, and they achieve it differently. Prompt agents get private networking directly. Hosted agents support BYO VNet, with each session running in a VM-isolated sandbox connected to your network — a stronger isolation story, because sessions do not share a runtime.

The design mistake is assuming that "no infrastructure to manage" implies "no isolation available", and pushing a team to hosted agents purely for network compliance. Prompt agents are explicitly covered.

Owning the data

Network isolation controls the path; bring your own resources controls the destination. The named examples are storage, Azure AI Search, and Azure Cosmos DB for conversation state.

This is the answer to a distinct requirement: the transcripts must live in a database we own, in our subscription, under our retention policy. It is different from store=false, which says do not persist at all, and different from tracing, which is telemetry shaped for diagnostics. A compliance stem usually contains exactly one of these three, and the wrong two are always plausible.

Identity, and the key-shaped hole

Per-agent Entra identity lets several agents in one project each authenticate as themselves to different downstream APIs, with access scoped per identity and no shared credential. It also reaches outward: "agent identities can authenticate to external MCP servers", and "OAuth On-Behalf-Of (OBO) passthrough is supported when configured".

None of it matters while key authentication is enabled, because "the key grants full access without role restrictions". A keyless design is therefore a sequence, not a switch: enable the identity, grant it a least-privilege role, move clients to a token credential while keys still work, verify in telemetry that no caller presents a key, and only then disable local authentication.

Worked Examples

Example 1 — four teams at a bank. Central IT owns networking, identity, and cost, and will not let teams create accounts. Team leads must create projects and publish agents. Developers build and test only.

Foundry Account Owner for central IT — creates accounts, manages models, cannot build. Foundry Project Manager for team leads — creates projects, builds, and meets the documented minimum for publishing. Foundry User for developers — the least-privilege build-and-test role. Owner/Contributor/Reader fails because Contributor and Reader carry no project data actions.

Example 2 — a regulated conversation record. The customer requires transcripts in a database they own, inside their own subscription, for retention and audit.

Bring your own resources, using their own Azure Cosmos DB for conversation state. store=false answers the opposite requirement — no persistence at all. Exporting traces to Application Insights answers neither, because telemetry is sampled and retained for diagnostics rather than as a record.

Example 3 — one project, several downstream APIs. Each agent must authenticate as itself, with no shared credential, and access scoped per agent.

A dedicated Entra agent identity per agent, with each API scoped to the corresponding identity. Passing an agent name in a header on a shared project identity is authentication by assertion — every agent still holds the same access, and the header is a claim the downstream API cannot verify.

Visual Explanations

Scopes and what they govern:

The keyless migration, in order:

Common Mistakes

Assigning Contributor to developers. It can create resources and cannot build in a project.

Assuming seniority maps to role. Account Owner administers and cannot build; Project Manager builds and cannot create accounts. That split is deliberate.

Assigning by display name in code. The roles were renamed; the IDs were not.

Reaching for Cognitive Services * roles or Azure AI Developer. Both are explicitly ruled out for Foundry work — the latter is scoped to Azure Machine Learning workspaces and Foundry hubs.

Creating a project per agent to isolate access. Agent scope exists for that.

Believing prompt agents cannot be network-isolated. Private networking is available for them.

Disabling local authentication first. It is the verification gate, not the opening move.

Provisioning an Azure OpenAI resource for a build that will need agents. Only /openai/v1 is served.

Practice Exercises

1. A service principal only calls one agent's endpoint. Name the role and the scope, and say what agent-scope assignments do not grant.
2. Which persona gets Foundry Account Owner, and what can that role not do?
3. A team needs network isolation but wants no infrastructure to operate. What is available to them?
4. Transcripts must be retained under the customer's control for seven years. Which mechanism, and what do store=false and tracing each solve instead?
5. A pipeline provisions projects and every new project fails at first use with an authorization error. Diagnose it and give the durable fix.

Summary & Concept Map

Infrastructure design here is four boundary decisions. Scope — resource for administration, project for building, agent to narrow endpoint access. Role — least privilege from Agent Consumer upward, assigned by GUID, avoiding Cognitive Services * and Azure AI Developer. Network — private networking for prompt agents, BYO VNet with per-session VM isolation for hosted agents. Data — bring your own storage, search, and Cosmos DB when the organization must own the record. Behind all of them sits identity, which only carries authorization on the Entra path — so keyless is a sequence ending, not beginning, with disabling local auth.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.