Reasoning Models and the Reasoning Effort Dial

What this slice covers

Some models in the Foundry catalogue behave differently from ordinary chat models: before answering, they spend time working through the problem, and that work consumes tokens you never see. This note covers what reasoning models are, the parameter that controls how hard they think, and the budgeting consequences that follow. It is a distinct kind of model component, and it is the one most likely to surprise someone who has only used standard chat completions.

A third kind of token

You already know about input tokens and output tokens. Reasoning models introduce a third: reasoning tokens, generated as the model breaks a problem apart, weighs approaches, and discards lines of attack that do not hold up.

Two properties make these tokens exam-worthy. They are invisible in the message content — you do not receive them as text. But they are not free: they take up room in the context window and they are billed at the output rate. So a request can produce a two-sentence visible answer and still bill like a long one.

The usage object returned with a response reports the exact count, which is how you move from guessing to measuring. On the Chat Completions surface the figure sits under completion_tokens_details.reasoning_tokens; on the Responses surface, under output_tokens_details.reasoning_tokens.

Reasoning effort

The reasoning_effort parameter tells a reasoning model how much thinking to do before answering. The supported values vary by model and span a range from none and minimal through low, medium, and high up to xhigh and max. Defaults also vary by model, which means a value you never set is still in force.

The scale is not arbitrary; each setting has a workload it suits:

• none — latency-critical work that gains nothing from deliberation, such as voice interactions, fast retrieval, and classification.
• low — efficient reasoning with a modest latency cost: tool use, planning, search, multistep decisions where speed and price matter.
• medium — a balanced starting point for most workloads involving planning or judgement.
• high — hard reasoning, complex debugging, deep planning, high-value tasks where quality outweighs latency.
• xhigh — deep research, asynchronous workflows, long agentic runs, justified by evaluation results.
• max — the most complex tasks; worth comparing against xhigh before adopting.

Importantly, the model adapts within whichever setting you choose, spending fewer tokens on easy inputs and more on hard ones. Effort sets the ceiling of ambition, not a fixed expenditure. Raising it generally means longer runtimes and more reasoning tokens.

Budgeting: the failure mode to know

Because reasoning tokens share the context window with your prompt and your answer, a poorly sized request can exhaust the window while the model is still thinking. The response then comes back marked incomplete — and it can do so before any visible output exists at all. You pay for the input and the reasoning and receive nothing usable.

Two defences follow. Check the status field on every response rather than treating an empty message as an empty answer. And reserve generous headroom while you are learning a workload's shape — the guidance is to leave at least 25,000 tokens for reasoning plus output until you have measured what your prompts actually consume, then tune the buffer down.

You cap total generation with the same output limits used elsewhere: max_completion_tokens on Chat Completions, max_output_tokens on Responses. Both cover reasoning tokens, visible tokens, and formatting tokens together.

Operational differences worth knowing

Reasoning models do not accept the same parameter set as other chat models, so code ported across families can break. Client timeouts also need rethinking: reasoning models warrant far longer allowances than non-reasoning ones, because the thinking happens before the first token is returned. And the raw chain of thought is not something to extract by side channels — where summaries of the model's reasoning are available, they are exposed through a dedicated reasoning-summary parameter.

Mistakes to avoid

• Assuming a reasoning model is simply a better model. It trades latency and cost for depth, which is the wrong trade for classification, extraction, or voice.
• Forgetting that the default effort applies when you send nothing. Defaults differ between models, and a model upgrade can change the behaviour of code you did not touch.
• Sizing a request on visible output alone. Invisible reasoning tokens occupy the same window and the same bill.
• Treating an empty response as a model failure rather than checking whether the request ran out of room.

What to carry forward

Reasoning models add a hidden token stream, a dial that governs how much of it is produced, and a real risk of paying for thinking that never reaches an answer. Choose the lowest effort that passes your evaluation, cap the output, and always read the status.