GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
FA
Fara 7B
Fara · gguf-org
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
Vision
AP
Apocrypha-7b
Apocrypha · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen3.6 35B A3B DFlash
Qwen · Anbeeld
Params386M
Context256K
Best fitBF16 1.5 GB
CodingEmbedding
LL
Llama3-8B-Instruct-Replete-Adapted
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
CodingTool use
CH
Chaotic Harmony 24B
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
LL
shisa v2.1 llama3.2 3b
Llama · mradermacher
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
GGUF
TE
TerminalHorizon 27B
Terminalhorizon · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
QW
qwen38 flash next v3
Qwen · nitinpanj
Params177B
Context256K
Best fitQ4_K_M 3.2 GB
GGUF
QW
Qwen3 4B Qwen3.6 plus Reasoning Distilled
Qwen · khazarai
Params4B
Context256K
Best fitQ8_0 6.0 GB
Reasoning
ZR
ZR1 1.5B
bartowski
Params1.78B
Context128K
Best fitBF16 4.3 GB
Coding
QW
Qwen3.5 4B Python Coder
Qwen · Abiray
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
Coding
GR
Grug 27b v1.1
Grug · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
ReasoningTool useVision
QW
ISO27001 Qwen2.5 0.5B Edge
Qwen · sallani
Params494M
Context32K
Best fitQ4_K_M 1.0 GB
GGUF
GP
Gpt Oss 20b Derestricted
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
GGUF
LL
Llama 3.3 8B Instruct 128K Jbliterated
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
SE
SEX ROLEPLAY 3.2 1B i1
Sex · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
Roleplay
LE
ignos_leoscorpius-greennode-platypus-7b-v1
Leoscorpius · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
MI
MiniMax H3 GGUF MiniStack
Minimax · woodfireind
Params20.1B
Context256K
Best fitQ4_K_M 4.2 GB
GGUF
LF
LFM2.5 1.2B JP
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
GGUF
NE
Next 1b
Next · mattritchey
Params1B
Context32K
Best fitQ4_K_M 1.5 GB
Coding
QW
X Coder RL Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Best fitQ3_K_M 5.8 GB
Coding
OP
q-bert_optimus-7b
Optimus · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
GR
granite 4.2 3b heretic
Granite · saidutta69
Params3.66B
Context128K
Best fitQ8_0 5.1 GB
ReasoningRoleplayTool use
LL
Llama3.2 24B A3B II Dark Champion INSTRUCT Heretic Abliterated Uncensored
Llama · mradermacher
Params18B
Context128K
Best fitIQ1_M 5.8 GB
Roleplay
AT
Atom 80b
Atom · mradermacher
Params80B
Context256K
Q4_K_M48.4 GB
Reasoning
QW
Qwen3.8 35B A3B Distill
Qwen · ngquocvinh
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
MI
Mixtral AI CyberRolePlay Orca
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Roleplay
DE
Deepseek Coder 1.3b Instruct
DeepSeek · mradermacher
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
Coding
MI
Ministral 3 3B Reasoning UNCENSORED Heretic
Mistral · mradermacher
Params3.43B
Context256K
Best fitQ6_K 4.2 GB
Reasoning
MI
nruaif_kimiko-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF

Other memory sizes

Keep browsing