Embeddings and Retrieval: Why Search Became a Text-Analysis Workload

Ask a group of newcomers to name text-analysis tasks and you will hear sentiment, summarisation, translation, entity extraction. Almost nobody says search. Yet retrieval is now one of the most common text workloads in production, and understanding why requires a concept that sits underneath all the others: the embedding. This note covers what embeddings are, why they turned search into an AI problem, and how retrieval-augmented generation reframes what a "knowledge" workload actually is.

The idea of an embedding

An embedding is a numeric representation of a piece of content — a sentence, a paragraph, a chunk of a document — arranged so that things with similar meaning end up numerically close together. Nothing about it is human-readable. Its entire purpose is to make similarity computable.

That one property dissolves a problem that keyword search could never solve. A user asking about "staff leaving the company" and a policy document headed "voluntary attrition" share no words at all, yet they mean the same thing. Keyword matching fails; comparing embeddings succeeds, because the two phrases land near each other in the representation.

Embeddings are a distinct model type, not a side effect of a chat model. In the Foundry Models catalogue, embeddings appear as their own inference task alongside things like chat completion and audio generation, and embedding models are called out as behaving differently from language models in several respects. Treating them as a separate class of tool is the correct mental model.

From embeddings to a vector index

A single embedding is not useful on its own. The workload emerges when you compute embeddings across an entire corpus and store them in a structure that can be searched by proximity. That store is the vector index, and building it is the ingestion half of the work. Foundry supports exactly this pattern — using vector indexes and retrieval-augmented generation over your own custom data to produce answers specific to your use case.

Notice the shape of the workload this creates. It has two halves that run at different times and have different characteristics. Ingestion is a bulk, offline, throughput-shaped job over your whole corpus. Query time is an interactive, latency-shaped job over one question. Learners who picture only the chat window miss the larger half of the engineering.

Retrieval-augmented generation in one paragraph

RAG is the pattern that connects the two. A user asks a question; the system embeds the question, retrieves the most similar chunks from the index, and hands those chunks to a generative model along with the question. The model answers using material it was given rather than material it memorised.

The benefit is not merely accuracy. It is currency — the index can be updated today without retraining anything — and it is attribution, because you know which passages were supplied and can cite them. It also means an organisation's private documents can inform answers without those documents ever becoming part of a model's weights.

Why ingestion quality dominates the outcome

Here is the part that separates people who have shipped RAG from people who have read about it: the quality of the answer is bounded by the quality of what went into the index. If your ingestion flattened a document into a wall of text, the retrieval step will surface incoherent fragments and the generative model will produce confident nonsense from them.

This is why content preparation is treated as part of the retrieval workload rather than as a preliminary. Preserving layout, keeping tables intact, capturing hierarchical sections, and describing figures all matter, because a chart or diagram that is never described is information the index simply does not contain. The same logic extends beyond text: audio and video can be brought into a search index too, once transcription and segmentation have produced something retrievable.

Reading retrieval scenarios correctly

A scenario is describing a retrieval workload when it mentions answering questions over an organisation's own material, when it complains that people cannot find things, or when it asks for answers grounded in specific documents rather than general knowledge. It is not a retrieval workload merely because a chatbot is involved — a chatbot with no corpus behind it is a plain generation workload.

Mistakes to avoid

• Confusing embeddings with generation. They are different model types serving different purposes; you usually need both.
• Treating ingestion as setup. It is most of the work and most of the risk.
• Assuming keyword search is obsolete. Exact identifiers, part numbers, and codes are often best matched literally, and serious systems combine both.
• Ignoring non-text content. Figures, charts, and diagrams carry information that never reaches the index unless something describes them.

What to carry forward

Retrieval belongs on your mental list of text workloads. Embeddings make meaning comparable, vector indexes make it searchable, and RAG makes it answerable — and the leverage lies far more in what you put into the index than in which chat model reads it back out.