Classify, Split, Route: Sorting Content Before You Extract It

The problem this solves

Real input is rarely one clean document per file. A mortgage package arrives as a single PDF containing an application form, a pay slip and a bank statement. A scanning operation produces a hundred-page file that is really forty invoices. A sports broadcast contains commercials, play and studio commentary. If your only tool is field extraction, every one of these forces a preprocessing step outside the service, written and maintained by you.

Content Understanding folds that step inside the analyzer. Classification and splitting happen in the same analyze operation as extraction, so one call can decide what a file contains, cut it into pieces, and send each piece to the analyzer that knows how to read it.

Two properties do all the work

Classification is configured by two things in the analyzer config. Content categories is a set of named categories, each with a description. Segmentation is a switch that decides whether the file is treated as one thing or many.

With segmentation off, the whole file is classified into exactly one category. That is the classify-only case, and it is useful when files are homogeneous but arrive mixed — routing each incoming file to the right extraction analyzer without splitting anything.

With segmentation on, the service divides the file into segments and classifies each one. For documents the boundaries come from the category descriptions combined with structure such as pages and sections; for video they come from the descriptions combined with visual cues such as shot changes. The response carries a segments array with each segment's boundaries and assigned category.

Routing is a property of the category

Each category may carry an analyzer identifier. When it does, the classified content object is passed to that analyzer automatically, and its results come back in the same response. When it does not, the content is categorised and nothing further happens — the split-only case, useful when you want page ranges and labels for a downstream system.

The referenced analyzer is linked rather than copied, so it keeps behaving as itself. This has a consequence people trip over: the models named on the parent analyzer are used only for segmentation and classification. Each sub-analyzer uses its own model configuration when it extracts. A performance or cost change made on the parent does not propagate into the extraction work.

There is also a switch to omit the parent content object from the response, so that a routing analyzer returns only the results of the analyzers it dispatched to. Without it, a routing call returns both the whole-file content and the per-segment results, and the response gets large quickly.

No training, but descriptions do the teaching

Classification here needs no training dataset. You define categories in the analyzer definition, up to two hundred of them for documents, and exactly one for video, where the mechanism is used for segmentation rather than choice. Each category has a name and a description, and the combined name and description is limited to about a hundred and twenty characters — short enough that the description must be a distinguishing signal, not an essay.

The description is what the model reasons over, so it should carry distinguishing marks: the common title of the document, a phrase that appears prominently, a structural giveaway. Two tax years of the same form are distinguished by the year printed at the top, and the descriptions should say so.

Two habits improve results markedly. Use semantic categories rather than visual templates: one invoice category, not one per supplier layout, because template variation within a semantic type is exactly what the service is built to absorb. And define an other category. If you do not, every file is forced into one of your named categories, including files that belong in none of them. In the generally available API, including that catch-all is called out explicitly as something you need to do.

Nesting, and the page as the unit

A category's analyzer can itself be a classifier. That gives hierarchical classification: sort into broad types, then sort each type into subtypes, then extract. Documents support five levels of this nesting, video supports two. It is the mechanism behind the composed prebuilt analyzers that classify and route tax and mortgage packages.

The constraint to remember is granularity. In the generally available behaviour, the minimum unit of document classification is a single page, and segments start at page boundaries. A page that contains the end of one form and the start of another cannot be split by the generally available API. The preview API adds in-page segmentation behind its own opt-in flag, for cases such as medical records or stacked tax schedules where two document types genuinely share a page — and the response then reports each segment's page range, category, confidence and position on the page.

How this shows up in practice

The shape of a well-built pipeline is usually: one routing analyzer at the front holding the categories, small purpose-built analyzers behind it, and the parent configured to return only the routed results. Reading a design like that, you should be able to say which analyzer classified, which extracted, and which model configuration applied at each step.