GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3-1.7B
Qwen · unsloth
Params1.72B
Context40K
Best fitBF16 4.9 GB
XX
T5 V1_1 Xxl
Xxl · city96
Params4.76B
Context512
Best fitQ6_K 7.6 GB
QW
Qwen3 0.6B Instruct
Qwen · Qwen
Params596M
Context40K
Best fitQ8_0 2.1 GB
QW
Parable Qwen3 8B Claude Fable 5
Qwen · AnkitAI
Params8.2B
Context40K
Best fitQ5_K_M 7.6 GB
GE
gemma 4 12B it qat
Gemma · unsloth
Params440M
Context256K
Best fitBF16 2.4 GB
TE
Tess-4.5-27B
Tess · migtissera
Params27B
Context256K
Best fitQ8_0 5.8 GB
QW
Parable Qwen3 4B Claude Fable 5
Qwen · AnkitAI
Params4B
Context40K
Best fitQ8_0 6.0 GB
MI
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Best fitF16 2.9 GB
CA
Carbon 8B
Carbon · HuggingFaceBio
Params8.26B
Context32K
Best fitQ5_K_M 7.9 GB
QW
Huihui Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_S 7.9 GB
BG
bge small en v1.5
BGE · unsloth
Params33M
Context512
Best fitF16 0.7 GB
MU
Muse Glimmer 30B
Muse · meta-models
Params27.9B
Context128K
Best fitQ4_K_M 2.3 GB
QW
Qwen3-Vl-8B-Instruct
Qwen · unsloth
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB
AG
Agents A1 4B
Agents · InternScience
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
MX
mxbai-embed-large-v1
Mxbai · ChristianAzinn
Params334M
Context512
Q4_K_M0.2 GB
CA
Carbon 3B
Carbon · HuggingFaceBio
Params3.45B
Context32K
Best fitBF16 7.8 GB
MI
MiniCPM5 1B Claude Opus Fable5 Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Best fitF16 2.9 GB
EX
Experiment26 7B passthrough 10slice
Experiment · babybirdprd
Params7B
Context—
Best fitQ4_K_M 0.5 GB
QW
Qwen3.5 9B
Qwen · bartowski
Params9.2B
Context256K
Best fitQ3_K_XL 7.9 GB
LF
LFM2.5 24B A2B Uncensored
Lfm · AtomicChat
Params24B
Context128K
Q4_K_M15.4 GB
QW
Qwen2.5 Coder 14B Instruct AWQ
Qwen · Qwen
Params29B
Context128K
Best fitQ2_K 7.9 GB
QW
Qwen3Vl 2b Instruct
Qwen · Qwen
Params1.72B
Context256K
Best fitF16 4.9 GB
SP
Spark X2.6 4B
Spark · XHToken
Params4.45B
Context1024K
Best fitQ8_0 6.1 GB
AG
Agents A1 4B
Agents · InternScience
Params4.21B
Context256K
Best fitQ4_K_M 4.3 GB
GE
gemma 4 E2B it qat q4 0
Gemma · google
Params4.63B
Context128K
Best fitQ4_0 4.4 GB
QW
Qwen3.8 27B Uncensored
Qwen · azukivc
Params27.3B
Context256K
Best fitQ8_0 5.8 GB
QW
Qwen3.6 35B A3B
Qwen · bartowski
Params35.5B
Context256K
Best fitQ8_0 3.2 GB
LL
Llama 3.2 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Best fitF16 7.9 GB
LA
Laguna S 2.1 APEX
Laguna · Myric
Params118B
Context1024K
Best fitQ8_0 1.9 GB
GE
gemma 4 26B A4B it qat
Gemma · unsloth
Params430M
Context128K
Best fitBF16 2.4 GB