GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
HE
Hemmingway 1.5
Hemmingway · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
ReasoningTool use
OC
Occamy 1.0 APEX I NanoPlus
Occamy · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
ReasoningTool useVision
BE
BetterWright K2 Horizon 7B Uno
Betterwright · mradermacher
Params9B
Context512K
Best fitQ3_K_S 6.0 GB
Tool use
QW
Qwen2.5 Coder 3B Instruct
Qwen · unsloth
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
Coding
LL
Llama3.1 8B ShiningValiant2
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
MI
MiniCPM V 4.7 1B
MiniCPM · NANI-Nithin
Params1.01B
Context256K
Best fitBF16 2.9 GB
Vision
AV
Avrik Lite 4B
Avrik · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
GGUF
GE
GELab Zero 4B Preview
Gelab · bartowski
Params4B
Context256K
Best fitQ8_0 6.0 GB
Vision
QW
MN 2407 DSK QwQify v0.1 12B
Qwen · bartowski
Params12.2B
Context128K
Best fitIQ2_S 6.0 GB
Reasoning
DO
weyaxi_dolphin2.1-openorca-7b
Dolphin · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Video Companions
Qwen · off-grid-ai
Params4.02B
Context256K
Best fitQ4_K_M 5.4 GB
CodingTool useVision
LL
Llama3Ternary 10b
Llama · CMSManhattan
Params9.64B
Context8K
Best fitTQ2_0 5.9 GB
GGUF
UN
Unlimited OCR 2
Unlimited · AtomicChat
Params2.94B
Context128K
Best fitQ8_0 4.1 GB
Vision
LL
rombos Llama3 8B Instruct Replete Adapted i1
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
CodingTool use
QW
Qwen2.5 Coder 7B
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ4_1 5.8 GB
Coding
MI
nondzu_mistral-7b-code-16k-qlora
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Coding
OR
Ornith 1.5 9B Heretic
Ornith · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
GGUF
CH
Chocolatine 14B Instruct DPO v1.2 i1
Chocolatine · mradermacher
Params14B
Context4K
Best fitIQ2_XXS 5.9 GB
GGUF
KE
Kev-4B Artemis 2.5GB
Kev · turintech
Params4.21B
Context256K
Best fitQ4_K_M 4.1 GB
ReasoningTool use
IN
INTELLECT 3
Intellect · bartowski
Params107B
Context128K
Q4_K_M71.4 GB
Reasoning
EM
embeddinggemma-2
Embeddinggemma · AtomicChat
Params271M
Context256K
Best fitBF16 2.7 GB
EmbeddingVision
EM
Embeddinggemma 300M Qat Q4_0 Unquantized
Embeddinggemma · ggml-org
Params303M
Context2K
Best fitQ4_0 1.0 GB
Embedding
QW
Qwen3.5 4B Heretic 2xMerge DavidAU
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
Reasoning
OR
Ornith 1.5 27B A3B CoderX MTP
Ornith · ManniX-ITA
Params26.2B
Context256K
Q4_K_M16.1 GB
CodingVision
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Best fitQ4_K 5.9 GB
GGUF
JI
jina embeddings v2 base code
Jina · second-state
Params160M
Context8K
Best fitF16 1.1 GB
CodingEmbedding
LL
Hermes 3 Llama 3.1 8B
Llama · legraphista
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
RoleplayTool use
QW
Qwen2.5 Coder 7B Instruct Heretic
Qwen · saidutta69
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
CodingTool use
LL
Hermes 3 Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
RoleplayTool use
GE
Gemma 3 4b It
Gemma · DavidAU
Params3.88B
Context128K
Best fitQ6_K 5.6 GB
GGUF

Other memory sizes

Keep browsing