GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
RP
RPBizkit v11 12B Base
Rpbizkit · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
YI
Yi 1.5 9B Chat 16K Abliterated
Yi · mradermacher
Params8.83B
Context16K
Best fitQ3_K_L 6.0 GB
MI
fpham_generate_question_mistral_7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
HO
K2-Horizon-375B-A23B
Horizon · bartowski
Params379B
Context512K
Q4_K_M237.8 GB
SP
Spark X2.6 1.7B
Spark · mradermacher
Params1.71B
Context1024K
Best fitQ6_K 2.4 GB
NA
calderaai_naberius-7b
Naberius · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LL
Llama 3 Patronus Lynx 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Best fitQ3_K_L 5.9 GB
LL
Llama 3.2 1B NSFW Open Erotic
Llama · Novaciano
Params1.5B
Context128K
Smallest1.0 GB
QW
Qwen 3.8 27b Abliterated
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
MI
Mistral Nemo Instruct 2407 Absolute Heresy
Mistral · mradermacher
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
NO
Nova-1.1-0.8B
Nova · mradermacher
Params1.01B
Context256K
Best fitQ5_K_M 1.7 GB
FI
Fimbulvetr 40 i1
Fimbulvetr · mradermacher
Params10.7B
Context4K
Best fitIQ2_M 5.8 GB
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
MI
models--automerger--T3qInex12-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
CH
Chaotic Order 24B V3
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Best fitQ8_0 1.0 GB
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Best fitQ4_K 5.9 GB
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Best fitQ8_0 6.0 GB
IR
Irene-RP-v2-7B
Irene · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Qwen2 1.5B Instruct
Qwen · Goekdeniz-Guelmez
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
MI
MiniMax M3
Minimax · bartowski
Params426B
Context1024K
Q4_K_M261.3 GB
SE
Seed OSS 36B Instruct
Seed · bartowski
Params36.2B
Context512K
Q4_K_M21.8 GB
QW
Qwen3 0.6B Abliterated
Qwen · bartowski
Params596M
Context40K
Best fitBF16 2.6 GB
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Best fitQ3_K_M 5.9 GB
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · crusoeai
Params8B
Context1024K
Best fitQ3_K_L 5.9 GB
SO
Solar Open 100B
Solar · mradermacher
Params103B
Context128K
Q4_K_M62.3 GB
MI
uukuguy_speechless-code-mistral-7b-v1.0
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
ZD
ZDTaichu5.0 9B
Zdtaichu · abenzerps
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Best fitQ8_0 1.9 GB