GGUF models that fit in 32 GB
198257 GGUF language models have at least one quant whose estimated memory use is 32 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
AE
AesCode 8B
Aescode · mradermacher
Params8.19B
Context256K
Best fitQ6_K 8.4 GB
JI
Jina Embeddings v5 Text Nano Text Matching
Jina · jinaai
Params212M
Context8K
Best fitF16 1.2 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · kamjin
Params7.5B
Context128K
Best fitQ8_K 10.0 GB
QW
Qwen3.6 35B A3B Uncensored MTP
Qwen · mecha-factory
Params35.5B
Context256K
Best fitQ4_K_M 22.9 GB
OR
Ornith 1.0 9B
Ornith · bartowski
Params8.95B
Context256K
Best fitBF16 19.5 GB
QW
JoyFox Qwen3.6 35B A3B RP Aggressive
Qwen · mradermacher
Params35.5B
Context256K
Best fitQ6_K 30.4 GB
LF
LFM3-VL-2B
Lfm · lmstudio-community
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
GR
Granite-4.0-H-Small
Granite · unsloth
Params32.2B
Context1024K
Best fitQ5_K_XL 29.0 GB
LL
LLaMA Mesh
Llama · bartowski
Params8B
Context128K
Best fitF16 17.6 GB
QW
Whittle Qwen 3.8 35B A3B
Qwen · mradermacher
Params35.5B
Context256K
Best fitQ6_K 30.5 GB
MI
Ministral 3 8B Instruct 2512
Mistral · bartowski
Params8.49B
Context256K
Best fitBF16 18.6 GB
BG
Bge Small En v1.5
BGE · ggml-org
Params33M
Context512
Best fitQ8_0 0.7 GB
QW
Qwen2.5 VL 32B Geometry
Qwen · mradermacher
Params32.8B
Context125K
Best fitQ6_K 29.5 GB
QW
UnifiedReward Edit Qwen35 27b
Qwen · mradermacher
Params26.9B
Context256K
Best fitQ8_0 31.2 GB
AE
AesCode 8B
Aescode · prithivMLmods
Params8.19B
Context256K
Best fitBF16 18.1 GB
LL
tulu v1 llama2 70b i1
Llama · mradermacher
Params69B
Context2K
Best fitIQ3_XS 31.5 GB
JA
Jan-nano-128k
Jan · Menlo
Params4B
Context128K
Best fitQ8_0 6.0 GB
PH
Phi 3 mini 128k instruct
Phi · QuantFactory
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
PH
Phi 3 Medium 128k Instruct
Phi · mradermacher
Params14B
Context128K
Best fitQ6_K 13.6 GB
DE
DeepSeek R1 Distill Qwen 7B Abliterated v2
DeepSeek · Melvin56
Params7.62B
Context128K
Best fitF16 16.2 GB
LF
LFM2.5 8B A1B DSpark Draft v1
Lfm · LiquidAI
Params328M
Context125K
Best fitF16 1.2 GB
AV
Avrik Pro 12B
Avrik · mradermacher
Params11.9B
Context256K
Best fitQ6_K 23.2 GB
SA
Sarashina2.2 3b Instruct v0.1
Sarashina · mmnga
Params3.36B
Context8K
Best fitQ8_0 5.4 GB
QW
Qwen3 VL Embedding 8B
Qwen · mradermacher
Params7.57B
Context256K
Best fitF16 16.9 GB
JA
NT Java 1.1B
Java · InfosysEnterprise
Params1.24B
Context8K
Q4_K_M0.8 GB
MA
manaca 1b base
Manaca · NANI-Nithin
Params1.72B
Context4K
Best fitBF16 4.4 GB
AI
airoboros l2 13b gpt4 2.0
Airoboros · TheBloke
Params13B
Context4K
Best fitQ8_0 21.0 GB
NE
NVIDIA-Nemotron-3-Nano-30B-A3B
Nemotron · vultr
Params31.6B
Context1024K
Best fitIQ3_M 27.4 GB
OP
openbuddy_openbuddy-zephyr-7b-v14.1
Openbuddy · TheBloke
Params7.28B
Context32K
Best fitQ8_0 9.3 GB
LO
LongWriter Zero 32B
Longwriter · bartowski
Params32.8B
Context32K
Best fitQ6_K_L 29.9 GB