Scaling and optimizing a Git repository
Design a strategy for scaling and optimizing a Git repository
Scalar
Scalar configures Git for large repositories, applying the settings and background maintenance that keep everyday commands fast as a repository grows. Its central techniques:
| Technique | Effect |
|---|---|
| Partial clone | Fetch commit and tree data, download file contents on demand |
| Sparse checkout | Materialise only the directories you actually work in |
| Background maintenance | Housekeeping runs on a schedule instead of stalling a command |
The insight behind both: most developers in a large repository need a small fraction of it. Partial clone avoids downloading history you will never open; sparse checkout avoids writing files you will never edit.
Cross-repository sharing
The alternative to one enormous repository is several, with sharing between them:
| Mechanism | Nature |
|---|---|
| Package feeds (Azure Artifacts, GitHub Packages) | Versioned, resolved by the build — the usual answer |
| Submodules | A pinned pointer to another repository |
| Pipeline repository resources | Check out another repository during the build |
Prefer a package over a submodule. A package has a version, a changelog and a resolution step; a submodule is a commit pointer that everyone forgets to update and that breaks clones when it drifts.
Choosing
Scalar makes one big repository workable. Splitting plus package sharing makes several small ones workable. The deciding factor is whether the code genuinely changes together — code that ships together belongs together, and code that does not should be a package.
Primary sources