GGUF models that fit in 24 GB
193369 GGUF language models have at least one quant whose estimated memory use is 24 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
ST
berkeley-nest_starling-lm-7b-alpha
Starling · TheBloke
Params7.24B
Context8K
Best fitQ8_0 9.3 GB
QW
Qwopus3.6 27B v2
Qwen · Jackrong
Params461M
Context256K
Best fitQ5_K_M 22.2 GB
QW
Qwen3 14B ARPO DeepSearch
Qwen · mradermacher
Params14.8B
Context40K
Best fitQ6_K 14.0 GB
JE
JEV 9B
Jev · prithivMLmods
Params8.95B
Context256K
Best fitBF16 19.5 GB
QW
Qwen3.8-27B-DFlash2-ternary-bonsai2-r3
Qwen · naklitechie
Params1.92B
Context256K
Best fitQ4_K_M 1.8 GB
QW
Qwen3.5 4B Heretic
Qwen · darrellbest
Params4.33B
Context256K
Best fitBF16 10.3 GB
MY
Mythos v2 9b
Mythos · fableforge-ai
Params9.2B
Context256K
Best fitF16 20.0 GB
HO
Holo 3.1 4B
Holo · mradermacher
Params4.84B
Context256K
Best fitF16 11.3 GB
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · HauhauCS
Params3.97B
Context1024K
Best fitQ8_K 12.2 GB
MI
devstral-small-2507
Mistral · mistralai
Params23.6B
Context128K
Best fitQ5_K_M 18.6 GB
TO
tokyotech llm Swallow MS 7b instruct v0.1
Tokyotech · mmnga
Params7.33B
Context4K
Best fitQ8_0 9.4 GB
LL
codellama_codellama-34b-hf
Llama · TheBloke
Params33.7B
Context16K
Best fitQ4_K_M 22.3 GB
QW
qwen2-7b-instruct
Qwen · Qwen
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma4 12B IT Abliterated
Gemma · DuoNeural
Params11.9B
Context128K
Best fitQ5_K_M 21.9 GB
XO
Xor 26B A4B
Xor · mradermacher
Params25.2B
Context256K
Best fitQ4_K_S 24.0 GB
QW
Qwen 35B A3B SignOfFour Coder
Qwen · pragmaticcs
Params34.7B
Context256K
Best fitQ4_K_M 22.6 GB
PR
Promethean Dawn 26B A4B
Promethean · mradermacher
Params26B
Context256K
Best fitQ4_0 23.5 GB
QW
Qwen3.5 9B Cold Fusion GAIN v1.0 Uncensored Heretic NEO MAX Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Best fitQ8_0 12.3 GB
QW
Qwen3.5 0.8B GGUF uncensored opus distill
Qwen · amkkk
Params752M
Context256K
Best fitQ4_K_M 1.4 GB
QW
Qwen2.5 Coder 32B
Qwen · lmstudio-community
Params32.8B
Context32K
Best fitQ4_K_M 22.5 GB
EM
Source Hf Embeddinggemma 2
Embeddinggemma · ngquocvinh
Params271M
Context256K
Best fitQ6_K 2.4 GB
LF
LFM2.5 24B A2B Uncensored Heretic
Lfm · mradermacher
Params24B
Context128K
Best fitQ6_K 22.9 GB
WI
WizardLM 30B Uncensored Guanaco SuperCOT 30b i1
Wizardlm · mradermacher
Params32.5B
Context2K
Best fitIQ2_S 24.0 GB
QW
Qwen3.8-27B
Qwen · kingjones777
Params26.9B
Context256K
Smallest1.7 GB
JA
Jais-3-32B
Jais · mradermacher
Params34.8B
Context512K
Best fitQ4_K_M 23.7 GB
DA
DanTagGen alpha
Dantaggen · KBlueLeaf
Params393M
Context512
Best fitF16 2.1 GB
QW
Qwen2.5 VL 32B Instruct Abliterated
Qwen · mradermacher
Params32.8B
Context125K
Best fitQ4_1 23.3 GB
GE
Gemma 4 E2B It Heretic Ara
Gemma · mradermacher
Params4.65B
Context128K
Best fitF16 10.4 GB
NA
VisionPsy Nano 460M Flash GGUFs
Nanovlm · qvac
Params409M
Context8K
Q4_K_M0.3 GB
QW
JetBrains Qwen3.8 3.6 27B Blend
Qwen · bartowski
Params27.3B
Context256K
Best fitQ5_K_M 23.6 GB