GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
KA
keynote-technology_kai-7b-beta
Kai · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
Mistral NeMo Minitron 8B Instruct
Mistral · bartowski
Params8.41B
Context8K
Best fitIQ3_M 5.8 GB
QW
Qwen3.8 27B DFlash2
Qwen · Anbeeld
Params1.92B
Context256K
Best fitBF16 4.5 GB
CO
CorticalStack neurotic crown clown 7b ties
Corticalstack · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
LL
Llama Phi 3 DoRA i1
Llama · mradermacher
Params3.82B
Context4K
Best fitQ4_K_M 6.0 GB
TI
py007_tinyllama-1.1b-intermediate-step-480k-1t
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
MI
models--automerger--CalmexperimentT3q-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
sciphi_sciphi-self-rag-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
QW
Qwen2.5 Coder 1.5B Shellsmith
Qwen · ajayk007
Params1.54B
Context32K
Best fitF16 3.8 GB
NO
nomic-embed-code
Nomic · nomic-ai
Params7.1B
Context32K
Best fitQ5_K_M 6.0 GB
US
UserLM 8b
Userlm · bartowski
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
JA
Jais-3-3.7B
Jais · IFM
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
RI
Rivermind 12B v1
Rivermind · bartowski
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
LF
Lfm2.5-1.2B-Thinking
Lfm · unsloth
Params1.17B
Context125K
Best fitBF16 3.9 GB
CH
MN CharThink 12B Base
Charthink · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
QW
Qwen3.8 Flash Next MTP
Qwen · quimmedes
Params3.88B
Context256K
Best fitQ4_0 3.7 GB
ON
OneJev 0.8B
Onejev · mradermacher
Params1.01B
Context256K
Best fitF16 2.9 GB
FU
FuseChat-Kunoichi-10.7B
Fusechat · mradermacher
Params10.7B
Context8K
Best fitIQ2_M 5.8 GB
TH
TheSpice 7b FT ExperimentalOrca
Thespice · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
DA
pocketdoc_dans-adventurouswinds-mk2-7b
Dans · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
OpenChat 3.5 7B Mixtral v2.0 i1
Mistral · mradermacher
Params7.24B
Context8K
Best fitQ4_K_M 5.9 GB
MI
nurtureai_openhermes-2.5-mistral-7b-16k
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
MiniCPM5-2B
MiniCPM · Mungert
Params2.52B
Context128K
Best fitBF16 5.9 GB
QW
Qwen3 16B A3B
Qwen · bartowski
Params16B
Context40K
Best fitIQ2_XXS 5.7 GB
QW
Qwen3.5-2B
Qwen · trubisky
Params1.88B
Context256K
Best fitIQ3_M 1.9 GB
AR
arco reasoner i1
Arco · mradermacher
Params514M
Context8K
Best fitQ5_K_S 1.3 GB
DE
Decka 4B
Decka · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen3 14B RefusalDirection ThinkingAware
Qwen · mradermacher
Params14.8B
Context40K
Best fitIQ1_M 5.7 GB