Deploying a Model: Serverless API, Managed Compute, and Instant Access

What this slice covers

Between choosing a model in the catalogue and calling it from code sits a step people rush past: deployment. This note covers what deploying actually means in Microsoft Foundry, the two deployment options and which models use each, the path that skips deployment altogether, and the resource-level decisions that quietly determine what you are allowed to deploy at all.

What deploying means

Foundry Models is the hub where you discover models. Discovery does not make a model callable. To make a model available for inference requests you deploy it — that is, you create an addressable deployment with a name, backed by a way of paying for and serving the traffic. When later code refers to a model by a deployment name rather than the vendor's model name, this is why.

Foundry picks the appropriate option for you based on the model you choose, so the real learning objective is not choosing between them by hand but understanding what each implies about cost, data handling, and capability.

Serverless API

Serverless API is the preferred path and the one that covers most of what you will meet. It serves all Foundry Models — both the models sold by Azure (all the Azure OpenAI models plus selected models from other top providers, billed through your Azure subscription, covered by Azure service-level agreements, and supported by Microsoft) and selected models from partners and the community, such as Anthropic and specific Mistral, Cohere, and Meta models.

Within serverless API you also choose a deployment type, and this is where several exam-relevant ideas live at once. Global Standard, Data Zone Standard, and single-region Standard differ in where your data is processed — globally, within a data zone such as the US, EU, or APAC, or in one region — which makes deployment type a compliance decision, not just a performance one. Provisioned throughput reserves capacity in provisioned throughput units for predictable, low-latency performance instead of paying per token. Batch exists for volume work that can wait.

Serverless deployments also carry built-in content filtering with customisable configurations, support keyless authentication through Microsoft Entra ID as the recommended option, and can be placed behind virtual network integration for private access.

Managed compute

The second option, currently in public preview, is managed compute: a managed GPU platform-as-a-service for open-source, partner, industry, and custom-weight models that need dedicated accelerators. You get no virtual machines or clusters to own — Foundry sizes the deployment, provisions the accelerators, and keeps the runtime patched, using curated inference runtimes and accelerator families including A100, H100, and MI300X.

Three properties distinguish it sharply from serverless. Billing is hourly per accelerator SKU rather than per token, with auto-scaling and scale-to-zero so an idle deployment stops costing money. Quota is granted per accelerator SKU per region through the Foundry quota process and is separate from Azure virtual machine quota — existing VM quota cannot be applied to it. And in public preview, content filtering is not available, which is a material difference if you assumed the safety layer follows the model everywhere.

Managed compute deployments are still reached through the same unified project endpoint, with the same authentication, networking, and SDK surface, on a managed-deployments route.

The path that skips deployment

There is a third possibility that is not a deployment option at all. Instant access, in preview, lets you call supported models by name and start running inference immediately, with no serverless or managed compute deployment created. For a learner exploring the catalogue, this removes the largest single piece of friction between reading about a model and seeing what it does.

The resource decision underneath

One setup choice constrains everything above it. A Foundry resource is the primary resource type for new projects and needs no AI hub. If you instead work from an older Azure OpenAI resource, the catalogue shows only Azure OpenAI models for deployment — you get the Azure OpenAI slice and nothing else, until you move to a Foundry resource. Learners who cannot find a model they read about are very often looking at the wrong resource type rather than the wrong region.

Mistakes people make

The first is assuming deployment is always required; instant access exists precisely for the try-it case. The second is treating deployment type as a performance dial and missing that it also decides where data is processed. The third is confusing quota systems — managed compute quota is not VM quota, and requesting more of one does not help the other. The fourth is assuming preview features carry production guarantees; managed compute is preview, offered without a service-level agreement, and Microsoft does not recommend it for production workloads. The fifth is forgetting that a model unavailable in your deployment list is usually a regional-availability or resource-type question, not a bug.