What fits in 24 GB of VRAM
Data updated . Last reviewed the same day, when this catalog snapshot was written.
Usable memory here is 22.8 GB, which is 95% of 24 GB. 35 of 49 catalog models fit at Q4, Q8, or FP16. 4 catalog GPUs have this much VRAM. 13 of the fits do not fit the 16 GB tier.
Models
Sorted by the highest quant that fits, then by headroom. The tok/s estimate uses GeForce RTX 4090 bandwidth only.
| Model | Best quant | Estimate | Headroom | Est. tok/s |
|---|---|---|---|---|
| Llama 3.2 1B Instruct | FP16 | 3.0 GB | 19.8 GB | 408 tok/s |
| Llama 3.2 3B Instruct | FP16 | 7.7 GB | 15.1 GB | 157 tok/s |
| Phi-3.5 mini | FP16 | 9.2 GB | 13.6 GB | 132 tok/s |
| Phi-4 mini | FP16 | 9.2 GB | 13.6 GB | 131 tok/s |
| Gemma 3 4B | FP16 | 10.3 GB | 12.5 GB | 117 tok/s |
| Mistral 7B Instruct v0.3 | FP16 | 17.4 GB | 5.4 GB | 69.5 tok/s |
| Qwen2.5 7B Instruct | FP16 | 18.3 GB | 4.5 GB | 66.2 tok/s |
| Gemma 4 E4B | FP16 | 19.2 GB | 3.6 GB | 63.0 tok/s |
| DeepSeek R1 Distill Llama 8B | FP16 | 19.3 GB | 3.5 GB | 62.8 tok/s |
| Llama 3.1 8B Instruct | FP16 | 19.3 GB | 3.5 GB | 62.8 tok/s |
| Qwen3 8B | FP16 | 19.7 GB | 3.1 GB | 61.5 tok/s |
| Ministral 3 8B | FP16 | 21.4 GB | 1.4 GB | 56.5 tok/s |
| Gemma 2 9B | FP16 | 22.2 GB | 0.6 GB | 54.5 tok/s |
| DeepSeek V4 Flash | Q8 | 4.5 GB | 18.3 GB | 5925 tok/s |
| gpt-oss-20b | Q8 | 12.3 GB | 10.5 GB | 484 tok/s |
| Gemma 3 12B | Q8 | 12.5 GB | 10.3 GB | 80.6 tok/s |
| Gemma 4 12B | Q8 | 13.2 GB | 9.6 GB | 79.6 tok/s |
| Mistral Nemo 12B | Q8 | 14.4 GB | 8.4 GB | 77.4 tok/s |
| Ministral 3 14B | Q8 | 15.7 GB | 7.1 GB | 70.2 tok/s |
| Qwen3 14B | Q8 | 17.0 GB | 5.8 GB | 64.2 tok/s |
| Phi-4 | Q8 | 17.3 GB | 5.5 GB | 64.7 tok/s |
| Qwen2.5 14B Instruct | Q8 | 17.3 GB | 5.5 GB | 64.2 tok/s |
| Mistral Small 3.2 24B | Q4 | 15.7 GB | 7.1 GB | 70.3 tok/s |
| Gemma 3 27B | Q4 | 16.5 GB | 6.3 GB | 60.9 tok/s |
| Gemma 2 27B | Q4 | 16.6 GB | 6.2 GB | 60.6 tok/s |
| Qwen3.8 27B | Q4 | 17.0 GB | 5.8 GB | 61.2 tok/s |
| Gemma 4 26B-A4B | Q4 | 17.3 GB | 5.5 GB | 404 tok/s |
| Qwen3.6 27B | Q4 | 17.4 GB | 5.4 GB | 59.9 tok/s |
| GLM-4.7-Flash | Q4 | 18.8 GB | 4.0 GB | 573 tok/s |
| Qwen3 30B-A3B | Q4 | 19.4 GB | 3.4 GB | 553 tok/s |
| Gemma 4 31B | Q4 | 19.7 GB | 3.1 GB | 55.0 tok/s |
| Qwen3 32B | Q4 | 21.9 GB | 0.9 GB | 51.0 tok/s |
| DeepSeek R1 Distill Qwen 32B | Q4 | 22.0 GB | 0.8 GB | 50.8 tok/s |
| Qwen2.5 32B Instruct | Q4 | 22.0 GB | 0.8 GB | 50.8 tok/s |
| Qwen3.6 35B-A3B | Q4 | 22.3 GB | 0.5 GB | 546 tok/s |
Next misses at Q4: Mixtral 8x7B (27.5 GB), Llama 3.3 70B Instruct (42.5 GB), DeepSeek R1 Distill Llama 70B (45.2 GB), Qwen2.5 72B Instruct (46.7 GB), gpt-oss-120b (63.1 GB).
GPUs at 24 GB
- Radeon RX 7900 XTX
24.0 GB vram · $999
- GeForce RTX 3090
24.0 GB vram · $1,499
- GeForce RTX 4090
24.0 GB vram · $1,599
- GeForce RTX 3090 Ti
24.0 GB vram · $1,999
Questions
- How many catalog models fit in 24 GB of VRAM?
- 35 of 49 fit at Q4 or better inside 22.8 GB usable (95% of 24 GB).
- Which catalog GPUs have 24 GB of VRAM?
- Radeon RX 7900 XTX, GeForce RTX 3090, GeForce RTX 4090, GeForce RTX 3090 Ti.
- Is the tokens/sec column a benchmark?
- No. It divides the GeForce RTX 4090's 1008 GB/s by active weight size. Other 24 GB cards are slower or faster than that.