GGUF models that fit in 32 GB

198257 GGUF language models have at least one quant whose estimated memory use is 32 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3-Coder-30B-A3B-Instruct
Qwen · ggml-org
Params30.5B
Context256K
Best fitQ4_K_M 19.3 GB
CodingTool use
GE
Gemma 4 26B A4B StyleTune V2
Gemma · mradermacher
Params26B
Context256K
Best fitQ6_K 31.8 GB
Roleplay
LF
LFM2 24B A2B
Lfm · LiquidAI
Params23.8B
Context125K
Best fitQ8_0 28.5 GB
GGUF
QW
Qwen3 VL 8B Instruct Abliterated v2.0
Qwen · mradermacher
Params8.19B
Context256K
Best fitF16 18.1 GB
Tool useVision
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitF16 19.5 GB
ReasoningRoleplayTool use
QW
Qwen3.8 27B Omnimerge v6
Qwen · ManniX-ITA
Params27.3B
Context256K
Best fitQ8_0 31.7 GB
Vision
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Best fitF16 16.2 GB
GGUF
QW
Mica v0.1 4B
Qwen · sky7350
Params4.84B
Context256K
Best fitBF16 11.3 GB
GGUF
UN
Undi95 Emerhyst 20B
Undi · RichardErkhov
Params920M
Context—
Best fitQ2_K 8.8 GB
GGUF
MI
Dolphin Mistral 24B Venice Edition
Mistral · douganger
Params23.6B
Context128K
Best fitQ8_0 26.9 GB
Vision
GE
Gemma 4 12B It Heretic
Gemma · igorls
Params11.9B
Context128K
Best fitQ8_0 26.1 GB
GGUF
MI
Ministral-3-14B-Instruct-2512
Mistral · unsloth
Params13.5B
Context256K
Best fitBF16 28.9 GB
Vision
BG
bge-small-zh-v1.5
BGE · CompendiumLabs
Params24M
Context512
Best fitF32 0.7 GB
Embedding
QW
Qwen3.5-35B-A3B
Qwen · unsloth
Params35.5B
Context256K
Best fitQ6_K 31.1 GB
Vision
QW
Ingot XIV 27B Q2
Qwen · dendriteholdings
Params26.9B
Context256K
Smallest5.9 GB
ReasoningTool use
OP
opendecider small
Opendecider · manjunathshiva
Params4.02B
Context256K
Best fitQ8_0 6.0 GB
Tool use
QW
LocalAI functioncall qwen2.5 7b v0.5 Q4 K M
Qwen · LocalAI-io
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GGUF
TE
Tessera-27B-B-CJK
Tessera · ZelligeAI
Params27.3B
Context256K
Best fitQ6_K_XL 28.0 GB
Vision
GE
Gemory 26B A4B
Gemma · UltimateIntent
Params25.2B
Context256K
Best fitQ6_K 31.2 GB
RoleplayVision
JA
Jan-v2-VL-high
Jan · janhq
Params8.2B
Context256K
Q4_K_M5.0 GB
Vision
ME
Medgemma-1.5-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Best fitBF16 9.4 GB
ReasoningVision
QW
Qwen_Qwen3.5 27B
Qwen · lmstudio-community
Params27B
Context256K
Best fitQ8_0 31.2 GB
Vision
NE
privacy-filter-nemotron
Nemotron · LocalAI-io
Params1.4B
Context128K
Best fitF16 3.4 GB
GGUF
RO
Rocinante X 12B v1
Rocinante · bartowski
Params12.2B
Context128K
Best fitBF16 26.3 GB
GGUF
NA
Nanbeige4.2 3B
Nanbeige · gkraker04
Params4.17B
Context256K
Best fitBF16 9.6 GB
GGUF
JA
Jan code 4b
Jan · janhq
Params4.4B
Context256K
Q4_K_M2.7 GB
Coding
LL
Meta-Llama-3-120B-Instruct
Llama · lmstudio-community
Params122B
Context8K
Best fitIQ1_S 31.2 GB
GGUF
LF
LFM2 24B A2B
Lfm · lmstudio-community
Params23.8B
Context125K
Best fitQ8_0 28.5 GB
GGUF
GE
Gemma 4 E2B It Ultra Uncensored Heretic
Gemma · llmfan46
Params4.65B
Context128K
Best fitBF16 10.4 GB
Vision
GL
GLM 4 9B 0414
GLM · bartowski
Params9.4B
Context32K
Best fitBF16 19.6 GB
GGUF

Other memory sizes

Keep browsing