Integrate generative workflows into applications by using Foundry SDKs and connectors

Once a model or agent works, the remaining problem is integration: which API surface the application calls, how it reaches other Azure resources without embedding credentials, how one agent definition serves many callers, and how it behaves when the service pushes back. Each has a documented answer, and the last one — retry behaviour — has a default that surprises people.

Why This Matters

The Responses API is the single entry point. Foundry consolidates model and agent invocation behind one API. Knowing that stops you looking for a separate agent-invocation path.

Connections keep credentials out of code. A connection is a first-class object linking a project to another resource. Applications reference it; they do not carry the key.

Structured inputs let one definition serve many callers. Vector stores, containers, and MCP endpoints can be overridden per request, which is how a single agent serves multiple tenants without duplication.

The SDK already retries — twice. That default composes badly with a retry loop you add yourself.

Prerequisites

• What a project endpoint is and how it differs from a model endpoint.
• Managed identity and DefaultAzureCredential as a credential source.
• What a tool definition looks like on an agent.
• Basic 429 handling and exponential backoff.

Learning Objectives

By the end of this lesson you will be able to:

1. Invoke models and agents through the Responses API.
2. Use connections so applications never carry downstream credentials.
3. Apply structured inputs to parameterise one agent per request.
4. Integrate MCP, OpenAPI, and Toolbox custom tools.
5. Configure retry behaviour without multiplying the SDK's own attempts.

Building Blocks

The Responses API. Foundry exposes model and agent invocation through a single API rather than separate paths per capability — the same surface carries plain completions, tool-using turns, and agent runs.

Connections. A connection links a project to another resource — a storage account, an Azure AI Search service, another AI resource — and holds the authentication. Application code references the connection by name; the credential lives in the platform. This is what makes managed identity end-to-end achievable rather than partly key-based.

Structured inputs. Three overrides are supplied at request time and take precedence over the agent's stored configuration:

Custom tools. MCP connects an agent to a Model Context Protocol server. OpenAPI turns an existing REST API into a callable tool from its specification — the shortest path from a service you already run to a tool an agent can use. A2A (preview) is agent-to-agent. Toolbox exposes a curated, versioned set on one MCP endpoint, with create → test → promote to default.

Authentication options for tools. Key, Entra (managed identity), OAuth on-behalf-of passthrough, and unauthenticated. OBO passthrough is the one to recognise: the tool acts as the signed-in user, so downstream permission checks apply to that user rather than to a service principal.

Retry behaviour. The SDK retries twice by default. Adding your own retry loop on top produces attempts multiplied by three. Set max_retries=0 when you implement custom retry.

Deep Dive

One agent, many callers

The instinct when a service becomes multi-tenant is to create an agent per tenant. Structured inputs exist so you do not have to.

Because file_search.vector_store_ids can be supplied at request time, one agent definition — one set of instructions, one tool list, one version history — can be pointed at tenant A's documents on one call and tenant B's on the next. The same applies to code_interpreter.container for file context and to mcp.server_label / server_url / headers for routing to a per-tenant or per-environment MCP endpoint, including per-caller headers.

The operational payoff is real: one definition to update, one version history to audit, one set of evaluation results. The governance obligation is equally real — because the caller supplies the store or endpoint, the caller must not be able to choose another tenant's. That check belongs in your application, not in the agent.

The same mechanism handles environments cleanly: identical agent, different MCP server_url per environment, no duplicated definitions drifting apart.

Retry: the default that multiplies

Quota questions and integration questions meet here.

The SDK retries twice by default. A developer seeing 429s who wraps calls in their own three-attempt backoff has not built three attempts — they have built up to nine, each consuming quota and lengthening the tail latency of an already-throttled request. The documented fix is to set max_retries=0 and own the policy completely.

Two related facts belong with it. Failed requests still count toward rate limits, so aggressive retrying makes throttling worse rather than better. And a temporary rate limit adjustment is detectable: when x-ratelimit-limit-tokens in the response is lower than your configured TPM, the service has applied a temporary reduction — the correct response is to back off, not to retry harder.

Choosing a tool integration

Three routes reach an external system, and the starting point decides.

OpenAPI is the shortest path when a REST API already exists with a specification: the spec becomes the tool definition. No new hosting, no new code.

MCP fits when a Model Context Protocol server exists or when you want one endpoint exposing several capabilities. Wrapped in a Toolbox it becomes versioned — create, test, promote to default — so an agent pinned to a version does not silently acquire capability when the catalog changes.

Azure Functions fits when logic must be written rather than exposed, and gives managed identity and the usual hosting controls.

The authentication choice cuts across all three, and OAuth on-behalf-of passthrough is the distinctive one. With OBO the tool acts as the signed-in user, so downstream systems apply that user's permissions. A scenario requiring the agent to see only what the current user may see, without reimplementing an access model, is describing OBO — the same instinct that makes remote SharePoint the right knowledge source when permissions must be inherited.

Streaming, state, and instrumentation

Three integration details round out a production application.

Streaming matters wherever a person is waiting: partial output improves perceived latency substantially, and reasoning models make it more important because time-to-first-token grows with thinking.

State is a decision, not a default. Hosted agents maintain session-level state persistence, and threads keep conversation context — but anything that must be retained under your control belongs in your own Cosmos DB. Traces are telemetry: sampled, retention-bound, and not a system of record.

Instrumentation is OpenTelemetry into Application Insights, and it is worth enabling from the first integration rather than after the first incident, because the trace is what turns a bad evaluation score into a diagnosed cause.

Worked Examples

Example 1 — multi-tenant document isolation. A SaaS assistant must answer from each customer's own document set, with one agent definition to maintain.

Structured inputs: supply file_search.vector_store_ids per request, pointing at the calling tenant's vector store. One definition, one version history, one evaluation baseline. The application must enforce that a caller can only name their own store — the platform honours what it is given.

Example 2 — retries making throttling worse. A team hits 429s, adds a three-attempt backoff, and sees throttling increase.

The SDK already retries twice, so each logical call now makes up to nine attempts, and failed requests still count toward the limit. Set max_retries=0 and own the policy. If x-ratelimit-limit-tokens is below the configured TPM, a temporary rate limit adjustment is in effect and backing off — not retrying — is correct.

Example 3 — an agent that must respect user permissions. An agent queries an internal REST API, and each user must see only their own records. The API already enforces permissions.

An OpenAPI tool from the existing specification, authenticated with OAuth on-behalf-of passthrough so the call is made as the signed-in user and the API's existing checks apply. A service-principal identity would see everything and force a permission model to be rebuilt inside the agent.

Visual Explanations

The integration surface:

How retries multiply:

Common Mistakes

Looking for a separate agent-invocation API. The Responses API is the single entry point.

Embedding downstream keys in application code. Use connections.

Creating an agent per tenant. Structured inputs parameterise one definition per request.

Trusting caller-supplied store or endpoint identifiers. The platform honours what it is given; your application must authorise it.

Adding retry without disabling the SDK's. Two default retries multiply your loop.

Retrying harder on 429. Failed requests still count; a low x-ratelimit-limit-tokens signals a temporary reduction.

Using keys where least privilege is required. Keys grant full access without role restrictions.

Building a new service when an OpenAPI spec already exists.

Practice Exercises

1. Name the three structured-input overrides and a use for each.
2. A team adds a three-attempt retry and throttling worsens. Explain and fix.
3. Which tool authentication makes the call act as the signed-in user, and when is that the answer?
4. Why is a key unacceptable where least privilege is required?
5. Where should conversation history live when it must be retained for compliance, and why not traces?

Summary & Concept Map

Integration is five decisions. Call through the Responses API, the single entry point for models and agents. Authenticate with Entra and a managed identity rather than keys, which grant full access without role restrictions, and reach downstream resources through connections so credentials never sit in code. Parameterise one agent definition per request with structured inputs — vector stores, containers, MCP endpoint and headers — authorising the caller's choice in your own application. Integrate external systems by the shortest available route: OpenAPI from an existing spec, MCP wrapped in a versioned Toolbox, or Azure Functions for new logic, with OAuth on-behalf-of where the call must run as the signed-in user. And set retry explicitly: the SDK already retries twice, failed requests still count, and max_retries=0 is the prerequisite to owning the policy yourself.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.