GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
KA
keynote-technology_kai-7b-beta
Kai · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Coding
MI
Mistral NeMo Minitron 8B Instruct
Mistral · bartowski
Params8.41B
Context8K
Best fitIQ3_M 5.8 GB
GGUF
QW
Qwen3.8 27B DFlash2
Qwen · Anbeeld
Params1.92B
Context256K
Best fitBF16 4.5 GB
GGUF
CO
CorticalStack neurotic crown clown 7b ties
Corticalstack · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
GGUF
LL
Llama Phi 3 DoRA i1
Llama · mradermacher
Params3.82B
Context4K
Best fitQ4_K_M 6.0 GB
GGUF
TI
py007_tinyllama-1.1b-intermediate-step-480k-1t
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
ReasoningVision
MI
models--automerger--CalmexperimentT3q-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
MI
sciphi_sciphi-self-rag-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
Coding
QW
Qwen2.5 Coder 1.5B Shellsmith
Qwen · ajayk007
Params1.54B
Context32K
Best fitF16 3.8 GB
Coding
NO
nomic-embed-code
Nomic · nomic-ai
Params7.1B
Context32K
Best fitQ5_K_M 6.0 GB
CodingEmbedding
US
UserLM 8b
Userlm · bartowski
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
JA
Jais-3-3.7B
Jais · IFM
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
ReasoningTool use
RI
Rivermind 12B v1
Rivermind · bartowski
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
GGUF
LF
Lfm2.5-1.2B-Thinking
Lfm · unsloth
Params1.17B
Context125K
Best fitBF16 3.9 GB
Reasoning
CH
MN CharThink 12B Base
Charthink · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
GGUF
QW
Qwen3.8 Flash Next MTP
Qwen · quimmedes
Params3.88B
Context256K
Best fitQ4_0 3.7 GB
GGUF
ON
OneJev 0.8B
Onejev · mradermacher
Params1.01B
Context256K
Best fitF16 2.9 GB
Vision
FU
FuseChat-Kunoichi-10.7B
Fusechat · mradermacher
Params10.7B
Context8K
Best fitIQ2_M 5.8 GB
GGUF
TH
TheSpice 7b FT ExperimentalOrca
Thespice · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
GGUF
DA
pocketdoc_dans-adventurouswinds-mk2-7b
Dans · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
MI
OpenChat 3.5 7B Mixtral v2.0 i1
Mistral · mradermacher
Params7.24B
Context8K
Best fitQ4_K_M 5.9 GB
GGUF
MI
nurtureai_openhermes-2.5-mistral-7b-16k
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Tool use
MI
MiniCPM5-2B
MiniCPM · Mungert
Params2.52B
Context128K
Best fitBF16 5.9 GB
CodingTool use
QW
Qwen3 16B A3B
Qwen · bartowski
Params16B
Context40K
Best fitIQ2_XXS 5.7 GB
GGUF
QW
Qwen3.5-2B
Qwen · trubisky
Params1.88B
Context256K
Best fitIQ3_M 1.9 GB
GGUF
AR
arco reasoner i1
Arco · mradermacher
Params514M
Context8K
Best fitQ5_K_S 1.3 GB
GGUF
DE
Decka 4B
Decka · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
ReasoningTool use
QW
Qwen3 14B RefusalDirection ThinkingAware
Qwen · mradermacher
Params14.8B
Context40K
Best fitIQ1_M 5.7 GB
Reasoning

Other memory sizes

Keep browsing