GGUF models that fit in 24 GB
193369 GGUF language models have at least one quant whose estimated memory use is 24 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
VI
Viking-33B
Viking · mradermacher
Params33B
Context4K
Best fitQ4_K_M 22.4 GB
YI
ehartford_dolphin-2_2-yi-34b
Yi · TheBloke
Params34.4B
Context16K
Best fitQ4_K_M 23.2 GB
OP
openchat_openchat-3.5-1210
Openchat · TheBloke
Params7.24B
Context8K
Best fitQ8_0 9.3 GB
MI
MiniCPM V 4_6 Thinking
MiniCPM · openbmb
Params752M
Context256K
Best fitF16 2.4 GB
CL
Clover-35B-A3B-Preview
Clover · Diffuspace
Params34.7B
Context128K
Best fitQ4_0 20.8 GB
HU
humanizer-12b
Humanizer · ngquocvinh
Params11.9B
Context256K
Best fitQ6_K 23.2 GB
AE
AesCode 8B
Aescode · mradermacher
Params8.19B
Context256K
Best fitQ6_K 8.4 GB
JI
Jina Embeddings v5 Text Nano Text Matching
Jina · jinaai
Params212M
Context8K
Best fitF16 1.2 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · kamjin
Params7.5B
Context128K
Best fitQ8_K 10.0 GB
QW
Qwen3.6 35B A3B Uncensored MTP
Qwen · mecha-factory
Params35.5B
Context256K
Best fitQ4_K_M 22.9 GB
OR
Ornith 1.0 9B
Ornith · bartowski
Params8.95B
Context256K
Best fitBF16 19.5 GB
QW
JoyFox Qwen3.6 35B A3B RP Aggressive
Qwen · mradermacher
Params35.5B
Context256K
Best fitQ4_K_M 22.9 GB
LF
LFM3-VL-2B
Lfm · lmstudio-community
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
GR
Granite-4.0-H-Small
Granite · unsloth
Params32.2B
Context1024K
Best fitIQ4_XS 23.3 GB
LL
LLaMA Mesh
Llama · bartowski
Params8B
Context128K
Best fitF16 17.6 GB
QW
Whittle Qwen 3.8 35B A3B
Qwen · mradermacher
Params35.5B
Context256K
Best fitQ4_1 23.6 GB
MI
Ministral 3 8B Instruct 2512
Mistral · bartowski
Params8.49B
Context256K
Best fitBF16 18.6 GB
BG
Bge Small En v1.5
BGE · ggml-org
Params33M
Context512
Best fitQ8_0 0.7 GB
QW
Qwen2.5 VL 32B Geometry
Qwen · mradermacher
Params32.8B
Context125K
Best fitQ4_K_M 22.5 GB
QW
UnifiedReward Edit Qwen35 27b
Qwen · mradermacher
Params26.9B
Context256K
Best fitQ5_K_M 22.0 GB
AE
AesCode 8B
Aescode · prithivMLmods
Params8.19B
Context256K
Best fitBF16 18.1 GB
LL
tulu v1 llama2 70b i1
Llama · mradermacher
Params69B
Context2K
Best fitIQ2_XS 23.5 GB
JA
Jan-nano-128k
Jan · Menlo
Params4B
Context128K
Best fitQ8_0 6.0 GB
PH
Phi 3 mini 128k instruct
Phi · QuantFactory
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
PH
Phi 3 Medium 128k Instruct
Phi · mradermacher
Params14B
Context128K
Best fitQ6_K 13.6 GB
DE
DeepSeek R1 Distill Qwen 7B Abliterated v2
DeepSeek · Melvin56
Params7.62B
Context128K
Best fitF16 16.2 GB
LF
LFM2.5 8B A1B DSpark Draft v1
Lfm · LiquidAI
Params328M
Context125K
Best fitF16 1.2 GB
AV
Avrik Pro 12B
Avrik · mradermacher
Params11.9B
Context256K
Best fitQ6_K 23.2 GB
SA
Sarashina2.2 3b Instruct v0.1
Sarashina · mmnga
Params3.36B
Context8K
Best fitQ8_0 5.4 GB
QW
Qwen3 VL Embedding 8B
Qwen · mradermacher
Params7.57B
Context256K
Best fitF16 16.9 GB