GGUF models that fit in 12 GB

171517 GGUF language models have at least one quant whose estimated memory use is 12 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Hermes 3 Llama 3.2 3b Base Fft Chatml Rerun 8xh100 E4 2024 11 24 4k Steps
Llama · NousResearch
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
RoleplayTool use
MI
dolphin 2.9.3 mistral nemo 12b
Mistral · dphn
Params12.2B
Context1000K
Best fitQ6_K 11.9 GB
CodingTool use
QW
Qwen3 VL Reranker 8B
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ8_0 10.4 GB
EmbeddingVision
OL
ollama3.1
Ollama · xzlinuxmodels
Params8B
Context128K
Best fitQ6_K 8.2 GB
GGUF
OR
Ornith1.0 9B 3MPER0RR Abliterated
Ornith · mradermacher
Params8.95B
Context256K
Best fitQ8_0 11.1 GB
ReasoningTool use
ON
OneJev 4B
Onejev · mradermacher
Params4.84B
Context256K
Best fitF16 11.3 GB
Vision
QW
Qwen3.6 35B A3B
Qwen · byteshape
Params34.7B
Context256K
Best fitIQ2_S 10.6 GB
Vision
QW
Qwopus GLM 18B Merged
Qwen · KyleHessling1
Params15.9B
Context256K
Best fitQ3_K_M 10.6 GB
ReasoningTool use
LF
LFM2.5 VL 3B Uncensored
Lfm · SC117
Params2.7B
Context125K
Best fitBF16 7.9 GB
Vision
PH
Phi 4 Mini Reasoning
Phi · lmstudio-community
Params3.84B
Context128K
Best fitQ8_0 5.7 GB
CodingReasoning
ST
berkeley-nest_starling-lm-7b-alpha
Starling · TheBloke
Params7.24B
Context8K
Best fitQ8_0 9.3 GB
GGUF
QW
Qwen3 14B ARPO DeepSearch
Qwen · mradermacher
Params14.8B
Context40K
Best fitQ4_1 11.2 GB
Tool use
JE
JEV 9B
Jev · prithivMLmods
Params8.95B
Context256K
Best fitQ6_K 8.9 GB
GGUF
QW
Qwen3.8-27B-DFlash2-ternary-bonsai2-r3
Qwen · naklitechie
Params1.92B
Context256K
Best fitQ4_K_M 1.8 GB
GGUF
QW
Qwen3.5 4B Heretic
Qwen · darrellbest
Params4.33B
Context256K
Best fitBF16 10.3 GB
Vision
MY
Mythos v2 9b
Mythos · fableforge-ai
Params9.2B
Context256K
Best fitQ8_0 11.4 GB
Tool use
HO
Holo 3.1 4B
Holo · mradermacher
Params4.84B
Context256K
Best fitF16 11.3 GB
ReasoningTool useVision
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · HauhauCS
Params3.97B
Context1024K
Best fitQ6_K 11.6 GB
GGUF
TO
tokyotech llm Swallow MS 7b instruct v0.1
Tokyotech · mmnga
Params7.33B
Context4K
Best fitQ8_0 9.4 GB
GGUF
QW
qwen2-7b-instruct
Qwen · Qwen
Params7.62B
Context32K
Best fitQ8_0 9.1 GB
GGUF
XO
Xor 26B A4B
Xor · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GGUF
PR
Promethean Dawn 26B A4B
Promethean · mradermacher
Params26B
Context256K
Q4_K_M17.2 GB
Roleplay
QW
Qwen3.5 9B Cold Fusion GAIN v1.0 Uncensored Heretic NEO MAX Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Best fitQ6_K 10.4 GB
CodingReasoningRoleplay
QW
Qwen3.5 0.8B GGUF uncensored opus distill
Qwen · amkkk
Params752M
Context256K
Best fitQ4_K_M 1.4 GB
ReasoningVision
EM
Source Hf Embeddinggemma 2
Embeddinggemma · ngquocvinh
Params271M
Context256K
Best fitQ6_K 2.4 GB
EmbeddingVision
LF
LFM2.5 24B A2B Uncensored Heretic
Lfm · mradermacher
Params24B
Context128K
Best fitQ2_K 12.0 GB
GGUF
QW
Qwen3.8-27B
Qwen · kingjones777
Params26.9B
Context256K
Best fitQ4_0 4.4 GB
Vision
JA
Jais-3-32B
Jais · mradermacher
Params34.8B
Context512K
Best fitIQ1_M 11.4 GB
Reasoning
DA
DanTagGen alpha
Dantaggen · KBlueLeaf
Params393M
Context512
Best fitF16 2.1 GB
GGUF
QW
Qwen2.5 VL 32B Instruct Abliterated
Qwen · mradermacher
Params32.8B
Context125K
Best fitIQ2_XXS 11.7 GB
Vision

Other memory sizes

Keep browsing