GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
FA
Fara 7B
Fara · gguf-org
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
AP
Apocrypha-7b
Apocrypha · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Qwen3.6 35B A3B DFlash
Qwen · Anbeeld
Params386M
Context256K
Best fitBF16 1.5 GB
LL
Llama3-8B-Instruct-Replete-Adapted
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
CH
Chaotic Harmony 24B
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
LL
shisa v2.1 llama3.2 3b
Llama · mradermacher
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
TE
TerminalHorizon 27B
Terminalhorizon · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
qwen38 flash next v3
Qwen · nitinpanj
Params177B
Context256K
Best fitQ4_K_M 3.2 GB
QW
Qwen3 4B Qwen3.6 plus Reasoning Distilled
Qwen · khazarai
Params4B
Context256K
Best fitQ8_0 6.0 GB
ZR
ZR1 1.5B
bartowski
Params1.78B
Context128K
Best fitBF16 4.3 GB
QW
Qwen3.5 4B Python Coder
Qwen · Abiray
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
GR
Grug 27b v1.1
Grug · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
QW
ISO27001 Qwen2.5 0.5B Edge
Qwen · sallani
Params494M
Context32K
Best fitQ4_K_M 1.0 GB
GP
Gpt Oss 20b Derestricted
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
LL
Llama 3.3 8B Instruct 128K Jbliterated
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
SE
SEX ROLEPLAY 3.2 1B i1
Sex · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
LE
ignos_leoscorpius-greennode-platypus-7b-v1
Leoscorpius · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
MiniMax H3 GGUF MiniStack
Minimax · woodfireind
Params20.1B
Context256K
Best fitQ4_K_M 4.2 GB
LF
LFM2.5 1.2B JP
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
NE
Next 1b
Next · mattritchey
Params1B
Context32K
Best fitQ4_K_M 1.5 GB
QW
X Coder RL Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Best fitQ3_K_M 5.8 GB
OP
q-bert_optimus-7b
Optimus · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GR
granite 4.2 3b heretic
Granite · saidutta69
Params3.66B
Context128K
Best fitQ8_0 5.1 GB
LL
Llama3.2 24B A3B II Dark Champion INSTRUCT Heretic Abliterated Uncensored
Llama · mradermacher
Params18B
Context128K
Best fitIQ1_M 5.8 GB
AT
Atom 80b
Atom · mradermacher
Params80B
Context256K
Q4_K_M48.4 GB
QW
Qwen3.8 35B A3B Distill
Qwen · ngquocvinh
Params35.5B
Context256K
Q4_K_M21.7 GB
MI
Mixtral AI CyberRolePlay Orca
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
DE
Deepseek Coder 1.3b Instruct
DeepSeek · mradermacher
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
MI
Ministral 3 3B Reasoning UNCENSORED Heretic
Mistral · mradermacher
Params3.43B
Context256K
Best fitQ6_K 4.2 GB
MI
nruaif_kimiko-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB