Deployment Options: Serverless API, Managed Compute, and Instant Access

What this slice covers

Choosing a model and using a model are separate acts. Between them sits deployment: the step that makes a catalogue entry callable from your code. Microsoft Foundry offers two deployment options plus one route that skips deployment entirely, and they differ in what infrastructure exists on your behalf, what you are billed for, and which models are even eligible. This note is about that choice. The finer-grained question of which type of serverless deployment to create is a separate slice.

Why a deployment exists at all

A model in the catalogue is a listing. To send it inference requests you deploy it, which creates something addressable with an endpoint, authentication, and an allocation of capacity. Nothing you read in the catalogue is callable until that has happened.

Crucially, the option is not a free choice you make from scratch: Foundry selects the appropriate deployment option based on the model you choose. Your job is to understand the consequences of the option a model lands in, not to negotiate it.

Serverless API

Serverless API is the preferred and most capable path, and it covers all Foundry Models — both the models sold by Azure and selected models from partners and community. Anthropic models and specific models from providers such as Mistral, Cohere, and Meta use it, as do all the Azure OpenAI models.

The defining trait is that Microsoft hosts the model. You never see a virtual machine. You get an API, and you are billed for what passes through it, typically in tokens, with pricing shown before you deploy.

What serverless API brings with it:

• Multiple deployment types, controlling where data is processed and how you pay.
• A choice of regional, data zone, or global processing to satisfy compliance requirements.
• Built-in content filtering with customisable configurations.
• Keyless authentication with Microsoft Entra ID, recommended over key-based auth, though both work.
• Private networking through virtual network integration.
• Provisioned throughput, where you reserve capacity in provisioned throughput units for predictable, low-latency performance.

Serverless API deployments live in Foundry resources — the primary resource type for new projects, requiring no AI hub. If you are working from an Azure OpenAI resource instead, the catalogue narrows to Azure OpenAI models only, and moving to a Foundry resource opens up the full set.

Managed compute

Managed compute is a managed GPU platform-as-a-service that runs open-source, partner, industry, and custom-weight models on dedicated GPU capacity. It is currently in public preview, which carries the usual consequences: no service-level agreement, and not recommended for production.

The point of managed compute is that you get dedicated accelerators without owning any of the plumbing. Foundry sizes the deployment, provisions the accelerators, and keeps the runtime patched. Deployments are sized in model-centric terms rather than by picking virtual machine SKUs, and they are served on the same unified Foundry project endpoint with the same authentication, networking, and SDK surface as everything else. Auto-scaling and scale-to-zero mean an idle deployment stops billing.

Two facts about billing and capacity are easy to get wrong. Billing is hourly per accelerator SKU, not per token. And quota is granted per accelerator SKU per region through the Foundry quota process, entirely separate from Azure virtual machine quota — existing VM quota cannot be applied to a managed compute deployment.

Note also that content filtering is not available in the public preview, which is a material difference from serverless API.

Instant access

There is a third route that is deliberately not a deployment type at all. Instant access, also in preview, lets you call supported models by name and start inferencing immediately. The same API, SDK, and client you already use continue to work; you simply pass an instant-access model name where a deployment name would go. It is the fastest path for prototyping and for trying an unfamiliar model, and it creates nothing in your subscription to manage or clean up.

Mistakes to avoid

• Believing you pick the deployment option. The model determines it.
• Assuming managed compute is the more powerful choice because it involves dedicated GPUs. The guidance is to use serverless API whenever possible; managed compute exists for models that require it.
• Planning a production rollout on managed compute or instant access while both are in preview without an SLA.
• Expecting existing Azure VM quota to cover managed compute. It does not.
• Creating a deployment just to try a model, when instant access would answer the same question with no resource to tear down.

What to carry forward

Serverless API is the default and the most capable. Managed compute is the dedicated-GPU path for open and custom models, in preview, billed by the hour. Instant access is the zero-setup route for evaluation. The model you select decides which of the first two applies.