Airllm

Seventy billion parameters on a single 4GB GPU is the claim AirLLM is built around. The usual wall is that large models demand memory in proportion to their size; AirLLM works on the inference path itself, trimming memory use until a modest card is enough. That puts state-of-the-art models within reach of people whose hardware would otherwise rule them out: developers, researchers and hobbyists experimenting with large language models, in software development and in research alike.