BrainyBeeBrainyBee
ExploreBlogStart Studying
HomeDeveloping AI Apps and Agents on Azure (AI-103)Integrate Foundry projects with CI/CD pipelines
Lesson2,637 words

Integrate Foundry projects with CI/CD pipelines

AI-103 › Unit 1: Plan and manage an Azure AI solution › Set up AI solutions in Foundry › Integrate Foundry projects with CI/CD pipelines

Integrate Foundry projects with CI/CD pipelines

Everything in a Foundry solution that can be created in a portal can also be defined in code — and the portal quietly does things for you that a pipeline does not. This lesson is about making Foundry a first-class part of a delivery pipeline, and about the conveniences that vanish when you stop clicking.

Why This Matters

Three properties make CI/CD integration worth designing deliberately.

Portal conveniences do not carry over. The automatic role assignment that makes a portal-created project work happens "when the project is created through the Microsoft Foundry portal UI", and "doesn't apply when deploying Foundry from SDK or CLI." A pipeline-provisioned project therefore starts unauthorized, and the failure surfaces at first use as an authorization error nobody changed anything to cause.

Identifiers move; contracts should not. The Foundry roles were renamed, and code assigning by display name broke while code assigning by role definition ID did not. The same lesson repeats with API versions and deployment names.

Versioning is already there, and it is not the same as your source control. Agent versions are "automatically snapshotted" by the platform, toolboxes version explicitly, and published agents get a stable endpoint. Understanding which layer versions what prevents building a parallel mechanism.

The exam frames this as pipeline failures with non-obvious causes.

"It works from the portal" is a diagnosis

When a stem says something works interactively and fails from a pipeline, look for a portal-only side effect — most often the automatic Foundry User assignment on a newly created project.

Prerequisites

  • The roles and scopes from earlier in this topic, and that RBAC only applies on the Entra path.
  • That agents come in two types, and prompt agents are configuration.
  • Basic CI/CD concepts: a service principal or workload identity running non-interactively.
  • The project endpoint format https://<resource-name>.services.ai.azure.com/api/projects/<project-name>.

Learning Objectives

By the end of this lesson you will be able to:

  1. Define prompt agents as code and explain what that enables.
  2. Diagnose the portal-only automatic role assignment and fix it in a pipeline.
  3. Assign roles durably using role definition IDs.
  4. Distinguish agent versioning, toolbox versioning, and publishing.
  5. Place CI/CD activity correctly within the agent development lifecycle.

Building Blocks

Code-first prompt agents. Prompt agents can be authored "in the Foundry portal for a quick start, or define them programmatically with the SDKs or REST API to integrate with your CI/CD workflows" — the code-first path "enabling version control, code review, and automated rollout."

The automatic assignment, and its limit. The documented minimum setup assigns Foundry User on the Foundry resource to your user principal and to your project's managed identity. Both "are added automatically when the project is created through the Microsoft Foundry portal UI" — and only if the creator can assign roles. It "doesn't apply when deploying Foundry from SDK or CLI."

Role identifiers. After the rename from Azure AI User / Owner / Account Owner / Project Manager, "use the role definition ID (GUID) instead of the role name in your code to avoid issues during the rename rollout" — Foundry User 53ca6127-db72-4b80-b1b0-d745d6d5456d, Foundry Owner c883944f-8b7b-4483-af10-35834be79c4a, Foundry Account Owner e47c6f54-e4a2-4754-9501-8e0985b135e1, Foundry Project Manager eadc314b-1a2d-4efa-be10-5d325db5065e.

Agent versioning and publishing. "As you iterate on your agent, versions are automatically snapshotted. Roll back to any previous version or compare changes between versions." Publishing "promote[s] an agent to a managed resource with a stable endpoint", and "published agents inherit the enterprise identity and access controls configured for your project."

Toolbox versioning. "Toolbox versioning gives you explicit control over when changes take effect — create a new version, test it, and promote it to default when you're ready."

The lifecycle. Seven stages: Create → Test → Trace → Evaluate → Optimize → Publish → Monitor, where Optimize means "automatically improve your hosted agent's instructions using the agent optimizer", and Test includes exercising MCP integrations in the playground "to validate tool connectivity, permissions, and behavior before publishing."

Three things that version, and what they cover

Attribute
Agent versions

The agent definition — instructions, model, tools

Automatic snapshots; roll back or compare

Toolbox versions

The shared tool set

Explicit: create, test, promote to default

Publishing

Where the agent is reachable

Managed resource with a stable endpoint

Deep Dive

Defining agents as code

The instinct that prompt agents are a portal artefact and hosted agents are the "real" code path is wrong, and it pushes teams onto container compute they do not need. Prompt agents defined through the SDK or REST get exactly what a delivery team wants — the definition lives in source control, changes go through review, and rollout is automated — while keeping the operational profile of a prompt agent: no runtime code, no compute to manage.

Hosted agents are the right choice when there is genuine custom code, not when the team wants CI/CD.

The failure every pipeline hits once

A pipeline creates the project successfully and the first application call fails with an authorization error. Nothing looks broken: the identity exists, the endpoint is right, the resource is there.

The cause is that the project's managed identity holds no role. An identity with no assignment authenticates successfully and is then denied, which is why the failure appears at first use rather than at provisioning. In the portal this never happens, because both assignments are made automatically — provided the creator can assign roles.

The fix is to assign explicitly in the pipeline: Foundry User on the Foundry resource to the project's managed identity, and to whatever principal the pipeline's tests run as. Then, because display names moved in the rename and IDs did not, assign by role definition ID.

Provisioning a project from a pipeline

  1. 1

    Create the resource and project

    Via SDK, CLI, or infrastructure-as-code.

Choosing durable identifiers

The rename is a specific instance of a general rule that shows up repeatedly on this exam: in automation, prefer the identifier that is contractual over the one that is presentational.

  • Roles — assign by GUID, not display name. The IDs and permissions were unchanged by the rename.
  • API versions — pin explicitly and test before moving. Output shapes change between versions, and the failure mode is a successful response with a changed representation rather than an error.
  • Deployments — address the deployment name you chose, which is stable across model version changes underneath it.
  • Toolboxes — pin the version if you must be able to say later what an agent was using.

Each of these turns a silent, delayed breakage into either a stable behaviour or a loud, immediate one.

What versions, and at which layer

Three mechanisms operate at different layers, and conflating them leads to building something redundant.

Agent versions are automatic and cover the agent definition. That is what lets an auditor establish which instructions governed a given answer — a property that matters well beyond delivery.

Toolbox versions are explicit and cover the shared tool set. The operational payoff is that a breaking change to a tool used by eight agents becomes a new version you test and then promote, instead of an edit that takes effect the moment it is saved. Editing in place makes the release window the test.

Publishing is not a version at all — it promotes an agent to a managed resource with a stable endpoint, inheriting the project's identity and access controls. It is the sixth of seven lifecycle stages, and notably not the last: Monitor follows it.

Where CI/CD sits in the lifecycle

Mapping the pipeline onto the seven stages clarifies what belongs where. Create and Test are pipeline stages — provisioning, definition, and validating tool connectivity and permissions before anything is published. Trace and Evaluate produce the evidence a gate can act on, which is how quality checks enter CI rather than being a manual review. Optimize uses the agent optimizer on the instructions, and it belongs after evaluation because it acts on what evaluation found. Publish is the release step. Monitor runs continuously afterwards, which is why a pipeline that ends at publish leaves the most important signal unowned.

A pipeline that ends at publish

Publishing promotes the agent; it does not tell you how it behaves afterwards. Monitor is the seventh stage — service metrics, dashboards, continuous and scheduled evaluation, and alerts on quality or harmful content. Treating release as the finish line is how six months of drift goes unnoticed.

Worked Examples

Example 1 — every new project fails at first use. A pipeline provisions projects; developers report an authorization error on the first call.

The automatic Foundry User assignment applies only to portal-created projects and does not apply from SDK or CLI, so the project's managed identity holds no role. Assign explicitly in the pipeline — and by role definition ID, so the assignment survives the display-name rename.

Example 2 — role assignment breaks after a platform update. A script assigns by display name and starts reporting the role as not found.

The Foundry roles were renamed from Azure AI User / Owner / Account Owner / Project Manager, and "the role IDs and core permissions are unchanged". Switch to the GUID. Re-creating the service principal would treat a lookup failure as a credential failure and change nothing.

Example 3 — a breaking tool change across eight agents. A shared toolbox needs a change that must be testable before any agent picks it up.

Create a new toolbox version, test it, and promote it to default. Editing in place makes the change live on save; a second toolbox plus repointing eight agents is eight configuration changes; detaching and reattaching is an outage by design.

Visual Explanations

Where the portal helps and a pipeline does not:

Loading Diagram...
Figure 1 — Mermaid diagram

The lifecycle, with the pipeline's share highlighted:

Loading Diagram...
Figure 2 — Mermaid diagram

Common Mistakes

Assuming prompt agents cannot be managed as code. They can, via SDK or REST, with version control, review, and automated rollout.

Relying on the portal's automatic role assignment in a pipeline. It does not apply from SDK or CLI.

Assigning roles by display name in automation. Names moved in the rename; GUIDs did not.

Treating an authorization failure as a credential problem. An identity with no role authenticates fine and is denied at first use.

Editing a shared toolbox in place. The change is live on save, so the release window becomes the test.

Building a parallel version history for agents. Versions are snapshotted automatically — use them.

Ending the pipeline at publish. Monitor is the seventh stage, and the one that catches drift.

Calling the latest API version automatically. Output shapes change between versions, and the failure is a successful response with a different representation.

Practice Exercises

  1. A team wants prompt agents under source control with review and automated rollout. Is that possible, and how?
  2. Projects provisioned by a pipeline fail at first use with an authorization error. Give the cause and the two-part fix.
  3. Why should automation assign Foundry roles by GUID?
  4. A toolbox shared by eight agents needs a breaking change, testable before adoption. Describe the process and say what is wrong with editing in place.
  5. Name the seven lifecycle stages in order and say which one a pipeline most often omits.
▶Answers
  1. Yes — define the prompt agent programmatically with the SDKs or REST API, the documented code-first path "enabling version control, code review, and automated rollout", while keeping no runtime code or compute.
  2. The automatic Foundry User assignment applies only to portal-UI-created projects and "doesn't apply when deploying from SDK or CLI". Fix: assign Foundry User explicitly to the project's managed identity, using the role definition ID.
  3. The roles were renamed (Azure AI User → Foundry User, and so on) while "the role IDs and core permissions are unchanged". Names are presentational; GUIDs are contractual.
  4. Create a new toolbox version, test it, then promote it to default. Editing in place takes effect the moment it is saved, so all eight agents get an untested change and the release window becomes the test.
  5. Create → Test → Trace → Evaluate → Optimize → Publish → Monitor. Pipelines most often omit Monitor, ending at publish.

Summary & Concept Map

Integrating Foundry with a pipeline is mostly about replacing portal side effects with explicit steps and choosing durable identifiers. Prompt agents are fully code-first, so CI/CD is not a reason to take on hosted agents. The automatic Foundry User assignment exists only for portal-created projects, which makes explicit role assignment a required pipeline step — by role definition ID, because the display names moved and the GUIDs did not. Three separate mechanisms version different things: agent definitions automatically, toolboxes explicitly with create-test-promote, and publishing which promotes to a stable endpoint rather than versioning anything. And the lifecycle does not end at publish — Monitor is the stage that catches what release cannot.

Loading Diagram...
Figure 3 — Mermaid diagram
Loading flashcards…

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.

All Developing AI Apps and Agents on Azure (AI-103) Study Resources

Related Notes

  • Choose an appropriate method for retrieval and indexing2,778 words
  • Quick Note — Choose an appropriate method for retrieval and indexing888 words
  • Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools3,097 words
  • Quick Note — Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools1,041 words
  • Choose appropriate memory, tool, and knowledge integration services for agent solutions2,815 words
  • Quick Note — Choose appropriate memory, tool, and knowledge integration services for agent solutions949 words
  • Choose the appropriate Foundry services for generative tasks, grounding, vector search, agent workflows, or multimodal processing2,733 words
  • Quick Note — Choose the appropriate Foundry services for generative tasks, grounding, vector search, agent workflows, or multimodal processing901 words
  • Apply responsible AI instrumentation, including evaluators, safety evaluations, and explanation tooling2,891 words
  • Configure safety filters, guardrails, risk detection, and content moderation2,795 words
  • Govern agent behavior with oversight modes, constraints, and tool-access controls2,863 words
  • Implement auditing through trace logging, provenance metadata, and approval workflows2,624 words

Ready to study Developing AI Apps and Agents on Azure (AI-103)?

Practice tests, flashcards, and all study notes — free, no sign-up.

Start Studying

Ready to study Developing AI Apps and Agents on Azure (AI-103)?

Practice tests, flashcards, and all study notes — free, no sign-up needed.

Start Studying — Free
Developing AI Apps and Agents on Azure (AI-103) ResourcesExplore All HivesBlogHome

© 2026 BrainyBee. Free AI-powered exam prep.

Loading Diagram...
Flowchart, top to bottom. How was the project created? connects to Foundry User assigned automatically<br/>to user principal AND project identity (Portal UI, creator can assign roles). How was the project created? connects to No automatic assignment (SDK or CLI). N connects to Identity authenticates, then DENIED at first use. F connects to Assign explicitly, by role definition ID.
Loading Diagram...
Flowchart, left to right. 1 Create connects to 2 Test. TE connects to 3 Trace. TR connects to 4 Evaluate. EV connects to 5 Optimize<br/>agent optimizer. OP connects to 6 Publish<br/>stable endpoint. PU connects to 7 Monitor<br/>continuous.
Loading Diagram...
Flowchart, top to bottom. CI/CD integration connects to Agents as code. CI/CD integration] --> AS[Agents as code connects to RBAC in the pipeline. CI/CD integration] --> AS[Agents as code connects to Versioning layers. CI/CD integration] --> AS[Agents as code connects to Lifecycle placement. AS connects to Prompt agents via SDK/REST<br/>version control, review, rollout. RB connects to Portal-only automatic assignment. RB connects to Assign explicitly from a pipeline. RB connects to By role definition ID, not name. 4 more statements.

CI/CD integration — retrieval

Card 1 of 6

Front of flashcard 1 of 6

Prompt agents as code

easy

Definable programmatically with the SDKs or REST API for CI/CD — "enabling version control, code review, and automated rollout" — while keeping no runtime code and no compute to manage.

cicdagents

CI/CD integration — retrieval

Card 1

Front

Prompt agents as code

Back

Definable programmatically with the SDKs or REST API for CI/CD — "enabling version control, code review, and automated rollout" — while keeping no runtime code and no compute to manage.

Card 2

Front

The portal-only role assignment

Back

Foundry User is assigned automatically to the user principal and the project's managed identity only when the project is created through the portal UI, and only if the creator can assign roles. It "doesn't apply when deploying from SDK or CLI".

Card 3

Front

Assign roles by GUID

Back

The Foundry roles were renamed; "the role IDs and core permissions are unchanged". Use the role definition ID in code — Foundry User is 53ca6127-db72-4b80-b1b0-d745d6d5456d.

Card 4

Front

Three versioning layers

Back

Agent versions — automatic snapshots of the definition. Toolbox versions — explicit: create, test, promote to default. Publishing — promotes to a managed resource with a stable endpoint (not a version).

Card 5

Front

Agent development lifecycle

Back

Create → Test → Trace → Evaluate → Optimize → Publish → Monitor. Optimize uses the agent optimizer on instructions and follows evaluation; Monitor follows publish and is the stage pipelines most often omit.

Card 6

Front

Durable identifiers in automation

Back

Prefer contractual over presentational: role GUIDs over names, pinned API versions over latest, your chosen deployment name, and a pinned toolbox version when provenance matters.

CI/CD integration — retrieval

Card 1

Front

Prompt agents as code

Back

Definable programmatically with the SDKs or REST API for CI/CD — "enabling version control, code review, and automated rollout" — while keeping no runtime code and no compute to manage.

Card 2

Front

The portal-only role assignment

Back

Foundry User is assigned automatically to the user principal and the project's managed identity only when the project is created through the portal UI, and only if the creator can assign roles. It "doesn't apply when deploying from SDK or CLI".

Card 3

Front

Assign roles by GUID

Back

The Foundry roles were renamed; "the role IDs and core permissions are unchanged". Use the role definition ID in code — Foundry User is 53ca6127-db72-4b80-b1b0-d745d6d5456d.

Card 4

Front

Three versioning layers

Back

Agent versions — automatic snapshots of the definition. Toolbox versions — explicit: create, test, promote to default. Publishing — promotes to a managed resource with a stable endpoint (not a version).

Card 5

Front

Agent development lifecycle

Back

Create → Test → Trace → Evaluate → Optimize → Publish → Monitor. Optimize uses the agent optimizer on instructions and follows evaluation; Monitor follows publish and is the stage pipelines most often omit.

Card 6

Front

Durable identifiers in automation

Back

Prefer contractual over presentational: role GUIDs over names, pinned API versions over latest, your chosen deployment name, and a pinned toolbox version when provenance matters.