GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Best fitQ6_K 6.0 GB
GGUF
HE
HelpingAI-15B
Helpingai · mradermacher
Params15.3B
Context4K
Best fitIQ2_XS 5.9 GB
GGUF
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Best fitQ4_K 6.0 GB
GGUF
PH
Athene Phi 3.5 mini instruct orpo
Phi · bartowski
Params3.82B
Context128K
Best fitQ4_K_M 6.0 GB
GGUF
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Best fitQ8_0 6.0 GB
Reasoning
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
ReasoningTool use
NE
NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash
Nemotron · ggml-org
Params7.52B
Context128K
Smallest0.6 GB
ReasoningTool use
LL
Llama Song Stream 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
GGUF
DE
Janus Pro 1B
DeepSeek · mradermacher
Params1.65B
Context16K
Best fitF16 5.4 GB
Vision
TR
TranslatePsy AfriSLM 2B
Translatepsy · mradermacher
Params2.39B
Context256K
Best fitQ6_K 2.9 GB
ReasoningTool use
BI
BioMistral 7B
Biomistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
EX
EXAONE 3.5 7.8B Instruct
Exaone · LGAI-EXAONE
Params7.8B
Context32K
Best fitIQ4_XS 5.9 GB
GGUF
OL
AMD OLMo 1B SFT DPO
OLMo · lmstudio-community
Params1.18B
Context2K
Best fitQ8_0 2.8 GB
GGUF
QW
Qwen3 Jan v1 256k Ctx 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context256K
Best fitQ4_K_M 6.0 GB
Coding
MI
Mistral 7B v0.1
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
GGUF
JI
JiuZhou
Jiuzhou · mradermacher
Params7.5B
Context32K
Best fitQ4_0 5.9 GB
GGUF
AL
AlphaMonarch 7B i1
Alphamonarch · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
GP
gpt2
Gpt · mradermacher
Params163M
Context1K
Best fitF16 1.1 GB
GGUF
LL
Reasoning Llama 1b v0.1
Llama · bartowski
Params1.24B
Context128K
Best fitF16 3.2 GB
Reasoning
LL
dolphin 2.9 llama3 8b 256k GGUF smashed
Llama · PrunaAI
Params8B
Context512K
Best fitQ3_K_L 5.9 GB
GGUF
QW
Qwen3.5 9B The Defiant Fable DARK ROAST Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
CodingReasoningRoleplay
CO
CodeQwen1.5-7B-Chat
Codeqwen · bartowski
Params7.25B
Context64K
Best fitQ4_K_M 5.8 GB
GGUF
QW
Qwen3 1.7B Coder Distilled SFT
Qwen · reaperdoesntknow
Params2.03B
Context40K
Best fitF16 5.5 GB
CodingReasoningTool use
ST
stable code instruct 3b
Stable · QuantFactory
Params2.8B
Context16K
Best fitQ6_K 5.5 GB
Coding
SM
Biggie SmoLlm 0.15B Base
SmolLM · mradermacher
Params181M
Context2K
Best fitF16 1.1 GB
GGUF
MA
MN 12B Mag Mell R1 Uncensored Scale1.2 i1
Mag · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
Reasoning
LF
LFM2.5 1.2B Nova Function Calling
Lfm · NovachronoAI
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
Tool use
DA
Darwin 36B Opus ABLITERATED HERETIC
Darwin · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B Orca DPO
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Tool use

Other memory sizes

Keep browsing