GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
MI
ehartford_dolphin-2.2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
QW
Qwen3 4B Abliterated
Qwen · mradermacher
Params4B
Context40K
Best fitQ8_0 6.0 GB
MI
MiMo V2.6 Flash RL NVFP4
Mimo · ProCreations
Params310B
Context1024K
Best fitBF16 4.9 GB
MI
MiniCPM V 4.7 35B A3B Abliterated
MiniCPM · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
LL
Llama3.1-8B-Chinese-Chat
Llama · shenzhi-wang
Params8B
Context128K
Best fitQ4_K_M 6.5 GB
DE
DeepSeek R1 0528 Qwen3 8B
DeepSeek · bartowski
Params8.19B
Context128K
Best fitQ5_K_L 7.9 GB
SP
rizzo flow
Spark · rizzoaiacademy
Params1.71B
Context1024K
Best fitQ8_0 6.1 GB
CO
Yi Coder 9B
Coder · RichardErkhov
Params8.83B
Context128K
Best fitQ5_1 7.9 GB
ER
Ernie-4.5-21B-A3B-Thinking
Ernie · unsloth
Params21.8B
Context128K
Best fitIQ1_S 8.0 GB
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
QW
Qwen3.5 27B Abliterated
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
DO
Dots3 Note Prev
Dots · bartowski
Params280B
Context512K
Q4_K_M171.1 GB
LL
Llama 3.2 1B Instruct
Llama · hugging-quants
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
DE
DeepHat V1 7B
Deephat · mradermacher
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
RO
RoMistral-7b-Instruct
Romistral · legraphista
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
QW
Dongfangshuo Qwen3.8 27B NSFW
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
YI
Yi 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Best fitQ8_0 7.5 GB
MI
Abliterated Dolphin3.0 R1 Mistral 24B
Mistral · mradermacher
Params23.6B
Context32K
Best fitIQ1_M 7.6 GB
QW
Qwen3.5 9B The Defiant Fable DARK ROAST Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ4_1 7.6 GB
LF
LFM2.5 Embedding 350M
Lfm · LiquidAI
Params354M
Context125K
Best fitBF16 1.7 GB
QW
shisa v2.1 qwen3 8b
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
JA
Jan-Nano-128K
Jan · unsloth
Params4B
Context128K
Best fitQ8_K_XL 6.8 GB
QW
Ingot 27B DFlash2
Qwen · dendriteholdings
Params1.92B
Context256K
Best fitBF16 4.5 GB
QW
Qwen3.8 27B Med Ablit Heretic
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
LL
turboderp Cat Llama 3 70B instruct
Llama · RichardErkhov
Params3.3B
Context—
Best fitIQ3_M 2.2 GB
PH
Phi 3.5 Mini instruct_Uncensored
Phi · bartowski
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
ZE
huggingfaceh4_zephyr-7b-beta
Zephyr · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
QW
Qwen3.8 27B Max Ablit Heretic
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
ST
stable code 3b
Stable · stabilityai
Params2.8B
Context16K
Best fitQ6_K 5.5 GB
GR
Granite-4.0-H-Micro
Granite · unsloth
Params3.2B
Context1024K
Best fitQ8_K_XL 7.0 GB