GGUF models that fit in 8 GB

151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
qwen3loop 0.6b sft deep supervision v1
Qwen · Lordnyx
Params596M
Context40K
Best fitF16 4.5 GB
ReasoningVision
DE
deepseek r1
DeepSeek · cortexso
Params70.6B
Context128K
Best fitQ3_K_S 7.4 GB
Reasoning
CA
CarbonVillain en 10.7B v4
Carbonvillain · TheBloke
Params10.7B
Context4K
Best fitQ3_K_L 7.8 GB
GGUF
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · bartowski
Params8B
Context128K
Best fitQ5_K_L 7.6 GB
GGUF
ZE
Zeta 2.1
Zeta · mradermacher
Params8.25B
Context32K
Best fitQ5_K_M 7.5 GB
GGUF
BR
Breeze-13B-32k-Instruct-v1_0
Breeze · mradermacher
Params12.7B
Context32K
Best fitQ3_K_S 8.0 GB
GGUF
QW
Qwen3 14B Abliterated
Qwen · mradermacher
Params14.8B
Context40K
Best fitIQ3_XXS 7.8 GB
GGUF
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated Heretic
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
Reasoning
OP
openPangu Embedded 7B V1.1
Openpangu · mradermacher
Params8.03B
Context32K
Best fitQ5_K_M 7.4 GB
Embedding
TH
TherapyBeagle 11B v1 i1
Therapybeagle · mradermacher
Params10.7B
Context32K
Best fitIQ4_XS 7.9 GB
GGUF
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Best fitQ6_K 7.1 GB
GGUF
EC
EchoVLM_v2
Echovlm · mradermacher
Params8.95B
Context256K
Best fitQ5_K_S 7.9 GB
ReasoningTool useVision
SY
maywell_synatra-7b-v0.3-base
Synatra · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
QW
GPT neo Custom 7B
Qwen · Szymon-Wachelko
Params7.6B
Context32K
Best fitQ6_K 7.2 GB
Coding
YI
EVA Yi 1.5 9B 32K V1
Yi · bartowski
Params8.83B
Context32K
Best fitQ5_K_L 7.7 GB
GGUF
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Best fitQ6_K 7.9 GB
GGUF
ER
ERNIE 4.5 21B A3B PT
Ernie · bartowski
Params21.8B
Context128K
Best fitIQ2_XS 7.3 GB
GGUF
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
GE
gemma 4 E4B it heretic
Gemma · igorls
Params7.5B
Context128K
Best fitQ5_K_M 7.7 GB
Vision
FA
MN 12B Faun RP RU
Faun · limloop
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
Roleplay
QW
Qwen3.5 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking
Qwen · FrosT-LyRo
Params39.1B
Context256K
Q4_K_M23.9 GB
CodingReasoningRoleplay
QW
Qwen3-Vl-2B-Thinking-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitBF16 4.9 GB
ReasoningVision
CA
CarbonVillain En 10.7B v4
Carbonvillain · mradermacher
Params10.7B
Context4K
Best fitIQ4_XS 7.9 GB
GGUF
LF
LMF 2.5 2B Code
Lfm · bunnycore
Params2.7B
Context128K
Best fitQ5_K_M 4.5 GB
Coding
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
GGUF
QW
Qwen3.8 9B Function Calling xLAM Unsloth
Qwen · ermiaazarkhalili
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
ReasoningTool use
TH
Thaki ASR KO Dialects v1 i1
Thaki · mradermacher
Params1.72B
Context64K
Best fitQ6_K 2.9 GB
GGUF
NO
nomic ai nomic embed code
Nomic · bartowski
Params7.1B
Context32K
Best fitQ6_K_L 6.9 GB
CodingEmbedding
AM
AmberChat
Amberchat · mradermacher
Params6.74B
Context2K
Best fitIQ3_M 7.9 GB
GGUF
EK
Ekbasis Skopos 27B
Ekbasis · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF

Other memory sizes

Keep browsing