Free fit check
How the math works
weights = parameters × bits per weight ÷ 8
KV cache = 2 × layers × KV heads × head size × bytes per value × tokens
total = weights + KV cache + your runtime allowance
- Parameters come from each model's official Hugging Face page (the safetensors total where the page shows one). These totals include vision parts, so a text-only file can be a little smaller.
- Bits per weight come from the GGUF block layouts in llama.cpp's ggml-common.h. For example, a Q4_K block stores 256 weights in 144 bytes: 144 × 8 ÷ 256 = 4.5 bits. Mixed files like Q4_K_M keep some tensors at higher precision, so they come out a bit bigger.
- KV cache uses each model's config.json. "2 ×" is one key and one value per token. Sliding-window layers only keep the last window of tokens (512 or 1,024 for Gemma 4). Qwen3.8's linear-attention layers keep a fixed-size state, which isn't counted here.
- Treat the KV number as an upper bound: Gemma 4 shares some keys, values and layers, which can make the real cache smaller.
- GB here means GiB (1,073,741,824 bytes), which is how most tools report memory.
The full fit sheet: $7, one time
- All eight models, including Gemma 4 31B, the E2B and E4B small models, Qwen3.8 Flash Next and GLM 5.3 Flash.
- The whole grid: every model × every quantization, with fit marks for your machine.
- KV cache by context length, from 4K up to each model's maximum, in f16 and q8_0.
- Copy-paste launch commands for llama.cpp's llama-server and vLLM, using only flags from their official docs.
- A cheat sheet for 8, 16, 24, 32, 64 and 128 GB machines: the best fit for each model.
After you pay, you'll land on your fit sheet page. Bookmark it.
These are estimates from published numbers. Real use varies with the runtime, drivers, batch size and how the file was made. They don't predict speed.
Why I made this
I'm Iggy, a software builder. Every time I tried a new local model I did this arithmetic on a napkin, and I usually forgot the KV cache until it ran out of memory at a long prompt. So I wrote the napkin down and put the sources next to it.