GGUF models that fit in 24 GB
193369 GGUF language models have at least one quant whose estimated memory use is 24 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
CA
CausalLM 14B DPO alpha
Causallm · tastypear
Params14.2B
Context8K
Best fitQ8_0 22.3 GB
GL
Glitter_Boy 2.0
Glitter · mradermacher
Params11.9B
Context128K
Best fitQ6_K 23.2 GB
PA
MS3.2 PaintedFantasy v4 24B
Paintedfantasy · bartowski
Params23.6B
Context128K
Best fitQ6_K_L 21.5 GB
OR
Ornith 1.5 35B A3B
Ornith · IsValorum
Params34.7B
Context256K
Smallest12.6 GB
MI
Roleplay-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
QW
flux2 klein 4B uncensored text encoder
Qwen · WeReCooking
Params4B
Context40K
Best fitQ4_0 4.1 GB
KA
KAT Coder V2.5 Dev 35B A3B MTP ABLITERATED
Kat · mradermacher
Params35.5B
Context256K
Best fitQ4_1 23.6 GB
GE
Gemma 4 26b A4B Heretic Styletune v2 Head
Gemma · mradermacher
Params26B
Context256K
Best fitQ4_0 23.5 GB
FI
Final Order 24B
Final · mradermacher
Params23.6B
Context128K
Best fitQ6_K 21.2 GB
GE
Gemma 4 12B Unified System One
Gemma · mradermacher
Params11.9B
Context256K
Best fitQ6_K 23.2 GB
QW
qwen3 coder 30b a3b compacted 19b 256k
Qwen · continuum-ai
Params19.7B
Context256K
Best fitQ8_0 22.2 GB
ST
G4 Starry Ocean 12B Heretic
Starry · mradermacher
Params11.9B
Context256K
Best fitQ6_K 23.2 GB
JI
jina embeddings v4 text matching
Jina · jinaai
Params3.1B
Context125K
Best fitF16 7.0 GB
MI
cognitivecomputations_dolphin-2.6-mistral-7b-dpo-laser
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ8_0 9.3 GB
HU
Huihui Hunyuan MT 7B Abliterated
Huihui · mradermacher
Params7.5B
Context256K
Best fitF16 16.6 GB
QW
Qwen3 4B Hivemind Instruct Heretic Abliterated Uncensored NEO Imatrix
Qwen · DavidAU
Params4B
Context256K
Best fitQ8_0 6.0 GB
QW
Qwen3.8 27B Uncensored Chinese
Qwen · mradermacher
Params27.3B
Context256K
Best fitQ5_K_M 22.2 GB
KE
Kev-9B Artemis 4.1GB
Kev · turintech
Params8.96B
Context256K
Best fitIQ3_M 6.4 GB
JA
Jan-v1-4B
Jan · janhq
Params4B
Context256K
Best fitQ8_0 6.0 GB
KA
Kainaticulous Rp 7b
Kainaticulous · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
QW
Qwen3 8B JP Uncensored
Qwen · ryo559
Params8.19B
Context40K
Best fitQ8_0 10.4 GB
GE
gemma-2-9b-it
Gemma · legraphista
Params9.24B
Context8K
Best fitBF16 21.8 GB
GL
GLM 4.6V
GLM · mradermacher
Params107B
Context128K
Q4_K_M70.4 GB
QW
Qwen3.5 abliterated MAX AIO
Qwen · prithivMLmods
Params752M
Context256K
Best fitBF16 18.8 GB
MU
Muse Glimmer 30B uncensored
Muse · TrevorJS
Params28B
Context128K
Best fitQ4_K_M 17.9 GB
GR
Granite 3b Code Base 2k
Granite · mradermacher
Params3.48B
Context2K
Best fitQ6_K 6.0 GB
GP
Huihui gpt oss 20b abliterated v2 MXFP4 MOE
gpt-oss · noctrex
Params20.9B
Context128K
Best fitMXFP4_MOE 13.0 GB
NY
Nyakura CausalLM RP 34B i1
Nyakura · mradermacher
Params34.4B
Context195K
Best fitQ4_K_M 23.2 GB
QW
Qwen3.8 27B TQ
Qwen · YTan2000
Params27.3B
Context256K
Smallest0.8 GB
LL
togethercomputer_llama-2-7b-32k-instruct
Llama · TheBloke
Params6.74B
Context32K
Best fitQ8_0 12.0 GB