GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Best fitQ6_K 6.0 GB
HE
HelpingAI-15B
Helpingai · mradermacher
Params15.3B
Context4K
Best fitIQ2_XS 5.9 GB
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Best fitQ4_K 6.0 GB
PH
Athene Phi 3.5 mini instruct orpo
Phi · bartowski
Params3.82B
Context128K
Best fitQ4_K_M 6.0 GB
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Best fitQ8_0 6.0 GB
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
NE
NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash
Nemotron · ggml-org
Params7.52B
Context128K
Smallest0.6 GB
LL
Llama Song Stream 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
DE
Janus Pro 1B
DeepSeek · mradermacher
Params1.65B
Context16K
Best fitF16 5.4 GB
TR
TranslatePsy AfriSLM 2B
Translatepsy · mradermacher
Params2.39B
Context256K
Best fitQ6_K 2.9 GB
BI
BioMistral 7B
Biomistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
EX
EXAONE 3.5 7.8B Instruct
Exaone · LGAI-EXAONE
Params7.8B
Context32K
Best fitIQ4_XS 5.9 GB
OL
AMD OLMo 1B SFT DPO
OLMo · lmstudio-community
Params1.18B
Context2K
Best fitQ8_0 2.8 GB
QW
Qwen3 Jan v1 256k Ctx 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context256K
Best fitQ4_K_M 6.0 GB
MI
Mistral 7B v0.1
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
JI
JiuZhou
Jiuzhou · mradermacher
Params7.5B
Context32K
Best fitQ4_0 5.9 GB
AL
AlphaMonarch 7B i1
Alphamonarch · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GP
gpt2
Gpt · mradermacher
Params163M
Context1K
Best fitF16 1.1 GB
LL
Reasoning Llama 1b v0.1
Llama · bartowski
Params1.24B
Context128K
Best fitF16 3.2 GB
LL
dolphin 2.9 llama3 8b 256k GGUF smashed
Llama · PrunaAI
Params8B
Context512K
Best fitQ3_K_L 5.9 GB
QW
Qwen3.5 9B The Defiant Fable DARK ROAST Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
CO
CodeQwen1.5-7B-Chat
Codeqwen · bartowski
Params7.25B
Context64K
Best fitQ4_K_M 5.8 GB
QW
Qwen3 1.7B Coder Distilled SFT
Qwen · reaperdoesntknow
Params2.03B
Context40K
Best fitF16 5.5 GB
ST
stable code instruct 3b
Stable · QuantFactory
Params2.8B
Context16K
Best fitQ6_K 5.5 GB
SM
Biggie SmoLlm 0.15B Base
SmolLM · mradermacher
Params181M
Context2K
Best fitF16 1.1 GB
MA
MN 12B Mag Mell R1 Uncensored Scale1.2 i1
Mag · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
LF
LFM2.5 1.2B Nova Function Calling
Lfm · NovachronoAI
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
DA
Darwin 36B Opus ABLITERATED HERETIC
Darwin · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
MI
OpenHermes 2.5 Mistral 7B Orca DPO
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB