GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
Gemma 3 270m It
Gemma · gguf-org
Params268M
Context32K
Best fitF32 1.7 GB
GGUF
QW
GPT 5 Distill Qwen3 4B Instruct
Qwen · Jackrong
Params4B
Context256K
Best fitQ8_0 6.0 GB
GGUF
LL
Impish_LLAMA_3B
Llama · l3utterfly
Params3.2B
Context128K
Best fitQ8_0 4.9 GB
GGUF
QW
Qwen3.5 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
ReasoningTool useVision
NA
NaNovel 9B
Nanovel · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
GGUF
US
UserRepGen 4B
Userrepgen · bartowski
Params4.21B
Context256K
Best fitQ6_K_L 5.4 GB
GGUF
OM
OmniCoder 9B
Omnicoder · bartowski
Params8.95B
Context256K
Best fitIQ3_XXS 5.8 GB
Coding
HU
HuatuoGPT 3 9B
Huatuogpt · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
Reasoning
LL
Trendyol LLM 8b Chat v2.0
Llama · bartowski
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
QW
Qwopus3 5 9B Coder Flash
Qwopus · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
CodingReasoningTool use
QW
Qwen3 VL 8B Thinking
Qwen · bartowski
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
ReasoningVision
QW
Qwen2.5 0.5B Instruct
Qwen · mradermacher
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.1 Nemotron Nano 8B v1
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
RU
RusMind.Моментум 3.5.1
Rusmind · DICPlay
Params4.65B
Context128K
Best fitQ4_K_M 4.5 GB
ReasoningTool use
LR
OrionLLM LRM 3.2
Lrm · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
ReasoningVision
MI
ehartford_samantha-1.2-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
LL
Herplete LLM Llama 3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
Tool use
SE
sciphi_sensei-7b-v1
Sensei · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
SM
SmolLlamix 8x101M take2
Smolllamix · mradermacher
Params399M
Context32K
Best fitQ8_0 1.0 GB
GGUF
MI
teknium_collectivecognition-v1.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
LO
MN LooseCannon 12B v2 i1
Loosecannon · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
GGUF
GI
GigaChat3 10B A1.8B
Gigachat · bartowski
Params10.7B
Context256K
Best fitQ3_K_S 5.9 GB
GGUF
QW
Qwen2.5 VL 3B uncensored bughunter v9
Qwen · mmp2055
Params3.1B
Context125K
Best fitQ4_K_M 2.7 GB
Tool useVision
AI
Aitana 7B S Instruct
Aitana · mradermacher
Params7.77B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
QW
MiMo V2.6 Distill Qwen 9B Ablit
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
Tool use
QW
open text detector qwen36 35b a3b oct10
Qwen · woog
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
QW
Qwen3-8B-abliterated
Qwen · Mungert
Params8.2B
Context40K
Best fitIQ3_M 5.7 GB
GGUF
QW
Qwen3 6B Hivemind Instruct Heretic Abliterated Uncensored NEO Imatrix
Qwen · DavidAU
Params5.94B
Context256K
Best fitQ4_K_M 6.0 GB
Roleplay
IN
models--automerger--Inex12Yam-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
GP
Gpt2 Large
Gpt · mradermacher
Params838M
Context1K
Best fitF16 3.7 GB
GGUF

Other memory sizes

Keep browsing