Shoehorn

Preset quantizations ignore the machine in front of you. Pick one that fits and you leave hundreds of megabytes of quality on the table; pick the next one up and you find out at load time that it does not fit. Shoehorn starts from the memory you actually have, subtracts what inference itself needs, and solves a per-tensor mixed-precision assignment against the remainder. The project reports landing within a rounding error of the budget, sometimes to the byte: 519.2 MiB of a 519.2 MiB budget, 13 KB of slack. It installs from a Homebrew tap. Apple Silicon Macs are supported, as is x86-64 Linux with an NVIDIA or AMD card. The site also carries a picker that scans Hugging Face's most-downloaded models for ones that fit a given budget, ranked by the quality that memory buys, and it runs entirely in the browser.