Mixtral 8x22B VRAM and RAM requirements

Large Mixtral. Q4 needs on the order of 85GB usable.

Total weights

141B

Active

39B

Context

66K tokens

License

Apache-2.0

mistralai/Mixtral-8x22B-Instruct-v0.1 on Hugging Face · 44,125 downloads on the Hub snapshot in this catalog

Memory

• Q4

  87.5 GB

• Q8

  151 GB

• FP16

  338 GB

• GGUF file size plus an FP16 KV-cache estimate at 8,192 tokens. The cache counts only the layers and heads named in the method on the About page.
• Formula estimate: parameter count × bytes per weight × 1.20. No GGUF file size is stored for this quant.

Compatible hardware

6 of 64 machines fit at Q4 or better. Fits means the estimate is inside the usable memory. Tight means less than 2 GB of room.

Amazon links are affiliate searches, not live prices.