GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
RP
RPBizkit v11 12B Base
Rpbizkit · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
GGUF
YI
Yi 1.5 9B Chat 16K Abliterated
Yi · mradermacher
Params8.83B
Context16K
Best fitQ3_K_L 6.0 GB
GGUF
MI
fpham_generate_question_mistral_7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
HO
K2-Horizon-375B-A23B
Horizon · bartowski
Params379B
Context512K
Q4_K_M237.8 GB
GGUF
SP
Spark X2.6 1.7B
Spark · mradermacher
Params1.71B
Context1024K
Best fitQ6_K 2.4 GB
ReasoningTool use
NA
calderaai_naberius-7b
Naberius · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Tool use
LL
Llama 3 Patronus Lynx 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Best fitQ3_K_L 5.9 GB
GGUF
LL
Llama 3.2 1B NSFW Open Erotic
Llama · Novaciano
Params1.5B
Context128K
Smallest1.0 GB
Tool use
QW
Qwen 3.8 27b Abliterated
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
MI
Mistral Nemo Instruct 2407 Absolute Heresy
Mistral · mradermacher
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
GGUF
NO
Nova-1.1-0.8B
Nova · mradermacher
Params1.01B
Context256K
Best fitQ5_K_M 1.7 GB
Tool use
FI
Fimbulvetr 40 i1
Fimbulvetr · mradermacher
Params10.7B
Context4K
Best fitIQ2_M 5.8 GB
GGUF
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
GGUF
MI
models--automerger--T3qInex12-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
CH
Chaotic Order 24B V3
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Best fitQ8_0 1.0 GB
GGUF
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Best fitQ4_K 5.9 GB
GGUF
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Best fitQ8_0 6.0 GB
GGUF
IR
Irene-RP-v2-7B
Irene · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Roleplay
QW
Qwen2 1.5B Instruct
Qwen · Goekdeniz-Guelmez
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
GGUF
MI
MiniMax M3
Minimax · bartowski
Params426B
Context1024K
Q4_K_M261.3 GB
Vision
SE
Seed OSS 36B Instruct
Seed · bartowski
Params36.2B
Context512K
Q4_K_M21.8 GB
GGUF
QW
Qwen3 0.6B Abliterated
Qwen · bartowski
Params596M
Context40K
Best fitBF16 2.6 GB
GGUF
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Best fitQ3_K_M 5.9 GB
Tool useVision
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · crusoeai
Params8B
Context1024K
Best fitQ3_K_L 5.9 GB
GGUF
SO
Solar Open 100B
Solar · mradermacher
Params103B
Context128K
Q4_K_M62.3 GB
GGUF
MI
uukuguy_speechless-code-mistral-7b-v1.0
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Coding
ZD
ZDTaichu5.0 9B
Zdtaichu · abenzerps
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
ReasoningVision
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
GGUF

Other memory sizes

Keep browsing