Monitoring pipeline health
Monitor pipeline health
Three signals, and each answers a different question.
| Metric | Question | Acted on by |
|---|---|---|
| Failure rate | Is the pipeline trustworthy? | Fixing the top failure cause |
| Duration | Is feedback fast enough? | Caching, parallelism, slicing |
| Flaky test rate | Can we believe a red build? | Quarantining or fixing the test |
Why flakiness is the one to watch
A pipeline with a high failure rate is annoying but honest. A pipeline with flaky tests is dishonest: it teaches the team that red means "re-run", so a genuine regression is retried away with everything else. Failure rate then looks fine while the safety net has quietly gone.
That is why flaky test rate is a first-class pipeline health metric rather than a testing detail.
Duration is about feedback, not cost
A 40-minute PR build does not merely waste agent minutes — it changes behaviour. Developers context-switch, batch changes into bigger PRs, and review less carefully. Optimising duration is optimising the team's working pattern.
Trend beats snapshot
A single run tells you almost nothing. Pipeline analytics exist because the signal is in the trend — duration creeping up, failure rate rising after a dependency change, one test that fails 3% of the time.
Primary sources