Configure model and agent deployments

Choosing a deployment type is one decision; configuring the deployment is another. This lesson covers what you assign, what that assignment actually controls, how to find capacity before you commit to a region, and the lifecycle traps that make quota disappear.

Why This Matters

Deployment configuration is where several counterintuitive mechanics live, and each one produces a failure that looks like something else.

Quota is held by assignment, not by usage. A forgotten test deployment keeps the TPM it was given. The symptom appears on a busy production deployment as throttling, and the cause is an idle one nobody has looked at.

Rate limiting and billing count different things. Rate limits apply to requests as they are received, using an estimate; usage metrics report billed tokens from successful requests. So a deployment can be throttled while its dashboards look comfortably under quota, and the explanation is arithmetic rather than a fault.

Deletion does not always release capacity. The portal enforces an ordering that the REST API does not, and the difference is a 48-hour window in which quota you believe you freed is unavailable.

The exam tests this as operational reasoning: given a symptom, name the mechanic.

Prerequisites

• The deployment types from the previous lesson, and that quota is per region, model, and deployment type.
• That a client addresses a deployment name, not a model name.
• Basic HTTP: status codes and response headers.
• That content filters exist and are applied to deployments.

Learning Objectives

By the end of this lesson you will be able to:

1. Assign TPM correctly and explain what it controls, including its relationship to RPM.
2. Choose between the Usages API and the Model Capacities API.
3. Diagnose the main 429 causes from their observable signatures.
4. Avoid the resource-deletion trap that withholds quota for 48 hours.
5. Associate a content filter with a deployment, and know where filters are created.

Building Blocks

Quota and TPM. "Quota is assigned to your subscription on a per-region, per-model, per-deployment-type basis in units of Tokens-per-Minute (TPM)." You assign TPM per deployment as you create it, and available quota falls by that amount. TPM "can be modified in increments of 1,000", and the assigned value "will map to the TPM and RPM rate limits enforced on your deployment."

The RPM ratio. RPM is "set proportionally to the TPM assignment", but "the ratio of Requests Per Minute (RPM) to Tokens Per Minute (TPM) for quota can vary by model" — one capacity unit is 6 RPM / 1,000 TPM on older chat models, and 1 RPM / 10,000 TPM on some reasoning models. When deploying programmatically "you don't have granular control over TPM and RPM as independent values."

What TPM is not. "Quota Tokens-Per-Minute (TPM) allocation isn't related to the maximum input token limit of a model."

Loosened limits. With TPM distributed across a subscription and region, "the maximum resources per region are increased to 30", and "the limit on creating no more than one deployment of the same model in a resource has been removed."

The two ARM APIs. The Usages API answers "How much of my quota have I consumed vs. my limit?", scoped to subscription plus location, input "Location only". The Model Capacities API answers "How much deployable capacity is available for a specific model?", scoped to "Subscription (all locations at once)", input "Model name, version, and format", typical use "Pre-check capacity before creating or scaling a deployment."

Rate-limit headers. x-ratelimit-limit-requests and -tokens (the ceiling), x-ratelimit-remaining-requests and -tokens (headroom), x-ratelimit-reset-* (window reset), and retry-after-ms — "Included in 429 responses. The recommended wait time (in milliseconds)."

Content filters. "Content filters can be configured at the resource level. Once a new configuration is created, it can be associated with one or more deployments." A request-level override is also available via the x-policy-id header, which "will override the deployment-level configuration, for the specific API call."

Deep Dive

What assigning TPM actually does

Assigning TPM does three things at once: it reduces available quota for that model, region, and deployment type; it sets the deployment's TPM rate limit; and it sets an RPM rate limit proportionally, at a ratio that depends on the model.

That last point matters for automation. A pipeline that assumes 6 RPM per 1,000 TPM will silently misallocate on a reasoning model where the unit is 1 RPM per 10,000 TPM — the deployment gets the tokens it asked for and a request rate it did not expect.

Because quota is held by assignment, capacity planning is about the whole estate rather than one deployment. Idle deployments keep their allocation, which is why "rebalance quota across deployments based on observed usage" is a documented remedy for throttling — often more effective than requesting an increase.

Reading a 429 correctly

There are four documented causes, and they are distinguishable from observable signals.

Genuine rate-limit exceeded — the error says "Rate limit is exceeded", usage is at your configured TPM/RPM. Remedy: raise the deployment's TPM, rebalance across deployments, or request an increase.

System capacity throttling — "The service is temporarily unable to process your request". Backend capacity is constrained and the condition is often transient. Retry with retry-after-ms; if persistent, consider provisioned throughput.

Temporary rate limit adjustment — 429s occur, configured quota is unchanged, and x-ratelimit-limit-tokens comes back lower than your configured TPM. The shared pool has protectively reduced your effective limit; it "typically resolves within a few hours". Retry with backoff and spread traffic.

Token budget exceeded by parameters — throttled while usage metrics look low. The estimate "includes the max_tokens parameter setting" and best_of, so a request reserving 4,000 tokens for a 200-token answer consumes that budget. Remedy: "Reduce max_tokens to match your expected response size", and set best_of to 1 unless you need it, because "each increment of best_of multiplies the token count against your rate limit."

The documented warning is worth internalising: "Many customers misinterpret capacity-related 429s as quota problems, leading to incorrect remediation."

Retrying without making it worse

Two facts combine badly. "Unsuccessful requests still count toward your per-minute rate limit. Continuously resending a request without backing off makes throttling worse." And the SDK already retries — the Python client "has built-in automatic retry with exponential backoff for 429 and transient errors", defaulting to two.

So a custom retry policy layered on the default multiplies: "each attempt from tenacity might itself trigger up to two additional SDK retries, leading to far more requests than expected." The instruction is explicit — "set max_retries=0 on the SDK client to disable its built-in retry" when using your own. Pick one layer, honour retry-after-ms, back off exponentially with jitter, and cap the attempts.

Finding capacity before committing

Use the Model Capacities API when the question is where. It takes a model name, version, and format and returns availability per location and deployment type across the subscription — which is exactly the pre-check before creating or scaling. Use the Usages API when the question is how much have we used here, for consumption monitoring and alerting. Teams scaling into a new region usually need both, and reaching only for Usages leaves them guessing about regions they have never deployed to.

The deletion trap

From the portal, deleting a resource "is blocked until the associated deployments are deleted", and deleting deployments first "allows quota allocations to be properly freed up."

Programmatically, that guard is absent: "if you delete a resource using the REST API or some other programmatic method, this bypasses the need to delete deployments first… the associated quota allocation will remain unavailable to assign to a new deployment for 48 hours until the resource is purged." The remedy is to "trigger an immediate purge for a deleted resource to free up quota."

The practical rule: cleanup automation should delete deployments explicitly before the resource, or purge afterwards. Otherwise the freed capacity is invisible for two days and the team concludes their quota calculation was wrong.

Filters, names, and versions

Three smaller configuration facts that recur.

Deployment name is the address. The client sends the deployment name you chose; it need not match the underlying model. Renaming at deployment time is a common source of confusion when a stem describes calls failing after a redeploy.

Content filters are resource-level objects associated with deployments. Creating one and expecting it to apply is the classic mistake — the association is the step that changes behaviour, and testing before it happens measures the default policy.

Agent versions are automatic. "As you iterate on your agent, versions are automatically snapshotted. Roll back to any previous version or compare changes between versions." Publishing then promotes an agent "to a managed resource with a stable endpoint".

Worked Examples

Example 1 — throttled with low usage. A summarization endpoint sets max_tokens to 4,000 while typical replies are 200 tokens. It takes 429s; Azure Monitor shows token usage well under quota.

The rate-limit estimate includes max_tokens, so each request reserves 4,000 tokens of budget regardless of what it generates. Usage metrics show billed tokens from successful requests, which is a different accounting. Reduce max_tokens to match the expected response size.

Example 2 — capacity that should be free. A cleanup script deletes unused resources via the REST API without deleting deployments first. The team then cannot create the production deployment they planned.

REST deletion bypasses the deployment check, so the quota stays unavailable for 48 hours until purge. Trigger an immediate purge — or, better, have the script delete deployments explicitly first.

Example 3 — planning a scale-out. The team must know where a specific model has deployable capacity, and separately track consumption against their approved limit.

Model Capacities API for the first — one model, all locations, per deployment type, designed as a pre-check. Usages API for the second — every quota line in a region with current value and limit. They answer different questions and the team needs both.

Visual Explanations

What one TPM assignment sets in motion:

Deletion paths and quota:

Common Mistakes

Assuming idle deployments release quota. Assignment holds it; rebalancing is the remedy.

Reading TPM as an input-token limit. It is unrelated to the model's maximum input tokens.

Assuming one RPM:TPM ratio. It varies by model, and programmatic deployment allocates in capacity units.

Requesting more quota for a capacity-related 429. The documented misdiagnosis — check x-ratelimit-limit-tokens against your configured TPM first.

Layering a custom retry on the SDK default. Set max_retries=0 on the client, or the attempts multiply.

Retrying without backoff. Failed requests still consume the limit, so aggressive retrying deepens the throttle.

Deleting resources via REST without purging. Quota is withheld for 48 hours.

Creating a content filter and not associating it. The association is what changes behaviour.

Practice Exercises

1. What three things does assigning TPM to a deployment set, and which of them varies by model?
2. Metrics show usage well under quota, yet the deployment is throttled. Name two request parameters that explain it.
3. x-ratelimit-limit-tokens returns lower than the configured TPM. What is happening and what is the right response?
4. A cleanup script deletes resources via REST; expected quota does not reappear. Explain and give the remedy.
5. Which API tells you where a specific model can be deployed, and which tells you what you have consumed in one region?

Summary & Concept Map

Deployment configuration is governed by three mechanics. Quota is assigned, so it is held by idle deployments and rebalanced rather than merely increased. Rate limits are enforced on requests as received using an estimate that includes max_tokens and best_of, which is why throttling and billed-token metrics disagree — and why failed requests still count, making unbacked-off retries self-defeating. And lifecycle matters: the portal enforces an ordering on deletion that REST does not, leaving quota withheld for 48 hours unless you purge. Around those sit the smaller facts that decide real incidents — the deployment name is the address, filters are resource-level objects that must be associated, and agent versions are snapshotted automatically.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.