Translate speech into other languages by using language models and Foundry Tools

Two services translate speech, and they differ in what comes out. Speech translation supports "speech-to-speech and speech-to-text translation" in real time. LLM speech's translate converts pre-recorded audio into text in a target language. If the requirement is spoken output, only one of them answers it.

Why This Matters

Speech-to-speech is a distinct capability. Speech translation "enables real-time, multilingual translation of speech" and is used "for speech-to-speech and speech-to-text translation".

LLM speech translate returns text. "Convert pre-recorded audio into text in a specified target language" — so no spoken output, and no streaming.

Language identification composes with translation. It works "by itself, with speech-to-text recognition, or with speech translation".

Prerequisites

• The three transcription modes.
• LLM speech's tasks and its preview and pre-recorded constraints.
• Translator's text translation features and terminology controls.
• SSML and voice selection for synthesis.

Learning Objectives

By the end of this lesson you will be able to:

1. Choose between speech translation and LLM speech translate.
2. Distinguish speech-to-speech from speech-to-text translation.
3. Compose language identification with translation.
4. Assemble a pipeline where no single service fits.
5. Apply terminology control across a speech translation flow.

Building Blocks

Speech translation. It "enables real-time, multilingual translation of speech to your applications, tools, and devices. Use this feature for speech-to-speech and speech-to-text translation." The scenario framing adds: "Generate high-quality speech-to-speech translation in real time, or automatically generate translated videos in a broad range of languages."

LLM speech translate (preview). "Convert pre-recorded audio into text in a specified target language." It brings "improved quality, deep contextual understanding, multilingual support, and prompt-tuning capabilities", and shares "the same ultra-fast inference performance as fast transcription".

Language identification. "Helps you identify languages spoken in audio by comparing them against a list of supported languages. Use language identification by itself, with speech-to-text recognition, or with speech translation."

Text-side terminology. Translator's dynamic dictionary and prevent translations on text translation v3; glossary files on synchronous document translation; Custom Translator dictionaries; and adaptive custom translation from 5–10,000 prealigned segment pairs (≤500 chars each) building "a custom bilingual index in minutes".

Synthesis side. Standard or custom voice, with SSML controlling "pitch, pronunciation, speaking rate, volume, and more" — the mechanism for correct rendering of names and terms in the target language.

Deep Dive

Output shape is the first question

The decisive property is whether the requirement needs spoken output.

Speech translation covers both directions: "use this feature for speech-to-speech and speech-to-text translation", generating "high-quality speech-to-speech translation in real time". So a live interpreter experience, a translated announcement, or a bilingual conversation is this service — and only this service, since LLM speech produces text.

LLM speech translate produces "text in a specified target language" from pre-recorded audio. That makes it a subtitling and transcript tool: captions for recorded video, translated meeting notes, an English record of a foreign-language voicemail.

Two eliminations follow immediately, and both appear in scenarios.

Live rules out LLM speech, because both its tasks take pre-recorded audio.

Spoken output rules out LLM speech, because its output is text. If spoken output is needed from a pre-recorded source, the pipeline is LLM speech or transcription for the text, then text to speech for the audio — an assembly, not a single call.

And preview rules out LLM speech under a production-only constraint, in which case fast transcription plus Translator is the fallback.

Composing language identification

Source language is frequently unknown, and language identification is designed to compose rather than stand alone: it works "by itself, with speech-to-text recognition, or with speech translation".

That gives three usable shapes.

Identify then route. Detect the language first and dispatch to a configured path — useful when different languages need different handling, voices, or downstream teams.

Identify alongside recognition. The transcript carries the language, which is what a multilingual archive needs for indexing and search.

Identify with translation. The translation step handles a source language it was not told about — the natural fit for an inbound channel where callers may speak any of several languages.

The related mechanism is multilingual transcription, which applies locale per phrase and is "enabled when no language/locale is specified or language is set to auto" — with the caution that unsupported locales fall back to "the closest supported locale, which is likely incorrect". So relying on omission is a decision; language identification is the deliberate alternative, and setting the locale explicitly is safest when the language is genuinely known.

Terminology across a translation flow

Names, products, and domain terms are where speech translation visibly fails, and control exists at three points.

At transcription. A custom speech model trained with acoustic, language, and pronunciation data fixes terms that are misheard in the source. This is upstream of translation and worth checking first: a term transcribed wrongly cannot be translated correctly.

At translation. For the text stage, Translator offers a dynamic dictionary for individual terms, prevent translations for spans that must be left untouched — product names and trademarks especially — glossary files on synchronous document translation, and adaptive custom translation, which builds "a custom bilingual index in minutes" from 5–10,000 segment pairs of ≤500 characters. In LLM speech, prompt-tuning plays this role.

At synthesis. SSML controls "pitch, pronunciation, speaking rate, volume", which is how a correctly translated name is correctly said in the target language.

The failure to anticipate: a brand name that survives transcription, gets helpfully translated into the target language, and is then pronounced as a common noun. Each stage has its own control, and fixing the wrong stage leaves the symptom in place.

Assembling a pipeline

Where no single service covers the requirement, the stages are transcription, translation, and synthesis — and assembling them is legitimate, with two caveats.

Errors compound. A mis-transcription becomes a mistranslation becomes a confidently mispronounced output, and each stage's mistake is invisible to the next. Instrument the stages separately, and fix upstream first.

Latency accumulates. For anything live, the sum of three stages is why speech translation exists as an integrated real-time capability — the same argument that makes Voice Live preferable to a hand-built conversational loop.

Assembly is right when the stages are genuinely separate: translate a recorded archive overnight, produce subtitles then optionally a dubbed track, or translate to text for records and separately synthesise for playback. It is the wrong default for a live bilingual conversation.

Worked Examples

Example 1 — a live bilingual conversation. Two speakers need to converse in different languages, hearing each other in their own.

Speech translation, which provides "real-time, multilingual translation of speech" and "speech-to-speech translation in real time". LLM speech translate is excluded twice: its output is text, and it operates on pre-recorded audio. Add language identification — which works "with speech translation" — if either speaker's language is not fixed in advance.

Example 2 — subtitles for recorded training video. Recorded modules need target-language subtitles, with control over domain terminology.

LLM speech translate — "convert pre-recorded audio into text in a specified target language" — with prompt-tuning for terminology, at "the same ultra-fast inference performance as fast transcription". It is preview, so under a production-only constraint use fast transcription plus Translator, applying prevent translations for product names and adaptive custom translation if a translation memory exists.

Example 3 — a dubbed track from recorded audio. The same modules need a spoken target-language track.

An assembly: translate to text — LLM speech translate, or transcription plus Translator — then text to speech for the audio, with SSML controlling pronunciation of names in the target language. LLM speech alone cannot do this, because its output is text. Alternatively, speech translation's speech-to-speech path covers the translated-video scenario directly.

Visual Explanations

Choosing by output and timing:

Terminology control at each stage:

Common Mistakes

Expecting spoken output from LLM speech translate. It returns text.

Using LLM speech for live translation. Its tasks take pre-recorded audio.

Relying on LLM speech in production. It is preview.

Forgetting speech translation does both directions. Speech-to-speech and speech-to-text.

Not composing language identification. It works with speech translation.

Fixing terminology at the wrong stage. Misheard, mistranslated, and mispronounced are three problems.

Translating product names. Use prevent translations.

Assembling three stages for a live conversation. Latency is the sum.

Practice Exercises

1. Which service supports speech-to-speech, and what does the other return?
2. Give two reasons LLM speech translate cannot serve a live interpreter scenario.
3. Name the three ways language identification composes.
4. A brand name is misheard, then translated, then mispronounced. Name the control at each stage.
5. What is the production-safe fallback for translated subtitles?

Summary & Concept Map

Speech translation splits by output shape and timing. Speech translation provides "real-time, multilingual translation of speech" for both speech-to-speech and speech-to-text, making it the only path to spoken output and the right one for live conversation and translated video. LLM speech translate converts pre-recorded audio into text in a target language, with prompt-tuning and fast-transcription-level performance — but it is preview, produces no audio, and cannot stream, so a spoken target from recorded source is an assembly: translate to text, then text to speech with SSML. Language identification composes "by itself, with speech-to-text recognition, or with speech translation" and is the deliberate alternative to multilingual transcription's likely-incorrect closest-locale fallback. And terminology has a control at each stage — custom speech for misheard, Translator's dictionaries and prevent-translations or prompt-tuning for mistranslated, SSML for mispronounced.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.