← All Tools
Local-Model VRAM Calculator
Will that model fit on this GPU? Pick a preset or punch in your own parameters, choose a quantization, set the context length, and see the breakdown: weights, KV cache, activations, total VRAM.
100% client-side. Pure arithmetic, no API calls, no telemetry.
Model
For GQA models (Llama 3, Mistral). MHA: equal to attention heads. MQA: 1.
Quantization & runtime
Estimated VRAM
—
Estimates are arithmetic upper-bound for inference. Real numbers vary ± ~10% across runtimes. For training, multiply weights by ~3-5× (gradients + optimizer states).
Common GPUs
| GPU | VRAM | Fit |
|---|
Math behind the numbers
- Weights = params × bytes-per-param. FP16 = 2 B/p, INT4 = 0.5 B/p, GGUF varies.
- KV cache = 2 (K and V) × batch × ctx × layers × kv-heads × head-dim × bytes-per-elem.
- Activations add ~7-15% on top depending on runtime; we use the runtime overhead slider.
- Long contexts are dominated by KV cache; quantising KV (INT8 / INT4) is the biggest lever.
- Multi-GPU reduces per-GPU memory, but inter-GPU comms add overhead. We don't model that — check vLLM / TGI docs.