What fits in 32 GB of VRAM
Data updated . Last reviewed the same day, when this catalog snapshot was written.
Usable memory here is 30.4 GB, which is 95% of 32 GB. 36 of 49 catalog models fit at Q4, Q8, or FP16. 1 catalog GPU has this much VRAM. 1 of the fits do not fit the 24 GB tier.
Models
Sorted by the highest quant that fits, then by headroom. The tok/s estimate uses GeForce RTX 5090 bandwidth only.
| Model | Best quant | Estimate | Headroom | Est. tok/s |
|---|---|---|---|---|
| Llama 3.2 1B Instruct | FP16 | 3.0 GB | 27.4 GB | 725 tok/s |
| Llama 3.2 3B Instruct | FP16 | 7.7 GB | 22.7 GB | 279 tok/s |
| Phi-3.5 mini | FP16 | 9.2 GB | 21.2 GB | 234 tok/s |
| Phi-4 mini | FP16 | 9.2 GB | 21.2 GB | 234 tok/s |
| Gemma 3 4B | FP16 | 10.3 GB | 20.1 GB | 208 tok/s |
| Mistral 7B Instruct v0.3 | FP16 | 17.4 GB | 13.0 GB | 124 tok/s |
| Qwen2.5 7B Instruct | FP16 | 18.3 GB | 12.1 GB | 118 tok/s |
| Gemma 4 E4B | FP16 | 19.2 GB | 11.2 GB | 112 tok/s |
| DeepSeek R1 Distill Llama 8B | FP16 | 19.3 GB | 11.1 GB | 112 tok/s |
| Llama 3.1 8B Instruct | FP16 | 19.3 GB | 11.1 GB | 112 tok/s |
| Qwen3 8B | FP16 | 19.7 GB | 10.7 GB | 109 tok/s |
| Ministral 3 8B | FP16 | 21.4 GB | 9.0 GB | 100 tok/s |
| Gemma 2 9B | FP16 | 22.2 GB | 8.2 GB | 96.9 tok/s |
| Gemma 4 12B | FP16 | 28.7 GB | 1.7 GB | 74.9 tok/s |
| Gemma 3 12B | FP16 | 29.2 GB | 1.2 GB | 73.5 tok/s |
| Mistral Nemo 12B | FP16 | 29.4 GB | 1.0 GB | 73.2 tok/s |
| DeepSeek V4 Flash | Q8 | 4.5 GB | 25.9 GB | 10533 tok/s |
| gpt-oss-20b | Q8 | 12.3 GB | 18.1 GB | 860 tok/s |
| Ministral 3 14B | Q8 | 15.7 GB | 14.7 GB | 125 tok/s |
| Qwen3 14B | Q8 | 17.0 GB | 13.4 GB | 114 tok/s |
| Phi-4 | Q8 | 17.3 GB | 13.1 GB | 115 tok/s |
| Qwen2.5 14B Instruct | Q8 | 17.3 GB | 13.1 GB | 114 tok/s |
| Mistral Small 3.2 24B | Q8 | 26.4 GB | 4.0 GB | 71.5 tok/s |
| Gemma 4 26B-A4B | Q8 | 27.2 GB | 3.2 GB | 453 tok/s |
| Gemma 3 27B | Q8 | 28.7 GB | 1.7 GB | 62.4 tok/s |
| Qwen3.6 27B | Q8 | 29.1 GB | 1.3 GB | 62.7 tok/s |
| Qwen3.8 27B | Q8 | 29.6 GB | 0.8 GB | 61.7 tok/s |
| Gemma 2 27B | Q8 | 30.0 GB | 0.4 GB | 59.6 tok/s |
| GLM-4.7-Flash | Q4 | 18.8 GB | 11.6 GB | 1018 tok/s |
| Qwen3 30B-A3B | Q4 | 19.4 GB | 11.0 GB | 983 tok/s |
| Gemma 4 31B | Q4 | 19.7 GB | 10.7 GB | 97.8 tok/s |
| Qwen3 32B | Q4 | 21.9 GB | 8.5 GB | 90.7 tok/s |
| DeepSeek R1 Distill Qwen 32B | Q4 | 22.0 GB | 8.4 GB | 90.3 tok/s |
| Qwen2.5 32B Instruct | Q4 | 22.0 GB | 8.4 GB | 90.3 tok/s |
| Qwen3.6 35B-A3B | Q4 | 22.3 GB | 8.1 GB | 970 tok/s |
| Mixtral 8x7B | Q4 | 27.5 GB | 2.9 GB | 245 tok/s |
Next misses at Q4: Llama 3.3 70B Instruct (42.5 GB), DeepSeek R1 Distill Llama 70B (45.2 GB), Qwen2.5 72B Instruct (46.7 GB), gpt-oss-120b (63.1 GB), Llama 4 Scout (65.4 GB).
GPUs at 32 GB
- GeForce RTX 5090
32.0 GB vram · $1,999
Questions
- How many catalog models fit in 32 GB of VRAM?
- 36 of 49 fit at Q4 or better inside 30.4 GB usable (95% of 32 GB).
- Which catalog GPUs have 32 GB of VRAM?
- GeForce RTX 5090.
- Is the tokens/sec column a benchmark?
- No. It divides the GeForce RTX 5090's 1792 GB/s by active weight size. Other 32 GB cards are slower or faster than that.