Optimising a pipeline
Optimize a pipeline for cost, time, performance, and reliability
Measure before optimising
The slowest stage is rarely where intuition points. Pipeline analytics show where duration actually goes; optimising anything else is effort spent for no feedback improvement.
The levers
| Lever | Effect |
|---|---|
| Caching | Skip dependency restore when the cache key matches |
| Parallelism | Split independent work across jobs (parallel by default) |
| Path filters | Do not build at all for documentation-only commits |
| Matrix | One definition, many legs, throttled by maxParallel |
| Artifacts over rebuild | Build once, download downstream |
| Self-hosted agents | Warm caches persist between runs |
| Right-sized agents | Bigger machines for genuinely CPU-bound work |
Caching and the key
A cache is only as good as its key. Key on the lockfile hash, so a dependency change invalidates the cache and an unrelated source change does not. Keying too broadly serves stale dependencies; keying too narrowly never hits.
Cost follows the billing model
With concurrency billing (Microsoft-hosted parallel jobs), splitting work across more jobs costs nothing extra provided you have the slots — it converts wall-clock time into parallel capacity. With per-minute billing, the same split may cost more, because total machine-minutes rise even as wall-clock falls.
That is the trade-off to state explicitly: parallelism buys time, and whether it costs money depends entirely on which model you are on.
Primary sources