GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Roblox Coder v2
Qwen · preferedev
Params4.33B
Context256K
Best fitQ5_K_M 4.8 GB
LL
Llama3.1 Heretic Unsensored
Llama · mradermacher
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
FR
FrogNano 4B 2609 Heretic
Frognano · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
EX
Experiment26 7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
NO
Noema 1.5 4B Deep
Noema · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen3.5 4B Anon Reasoning v2
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
OR
Ornith 1.0 35B Uncensored Heretic
Ornith · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
PY
Pythia 160m Deduped
Pythia · mradermacher
Params162M
Context2K
Best fitIQ1_S 0.9 GB
SU
SummLlama3.1 8B
Summllama · mradermacher
Params8.03B
Context128K
Best fitIQ4_XS 6.0 GB
LL
Llama 3.2 1B Instruct Abliterated
Llama · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
FI
Fin R1
Fin · mradermacher
Params7.6B
Context32K
Best fitQ4_K_M 5.7 GB
CH
Chocolatine 3B Instruct DPO v1.2
Chocolatine · bartowski
Params3.82B
Context128K
Best fitQ3_K_XL 5.9 GB
LL
Llm Jp 3 3.7b
Llm · mmnga
Params3.78B
Context4K
Best fitQ5_0 6.0 GB
AP
Apertus 70B Instruct 2509 Abliterated
Apertus · mradermacher
Params70.6B
Context64K
Q4_K_M43.7 GB
QW
Huihui Qwen3.5 4B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
PH
Phi 4 Reasoning plus_Nemotron Reasoning 0803_SFT GRPO
Phi · mradermacher
Params14.7B
Context32K
Best fitIQ1_M 5.8 GB
IN
InternLM2
InternLM · legraphista
Params1.89B
Context8K
Best fitBF16 5.1 GB
QW
imajev 4b
Qwen · mindchain
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
HU
Huihui OmniCoder 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
QW
Qwen Image Edit Rapid AIO NSFW v23 NVFP4
Qwen · FreedomAISVR
Params20.4B
Context32K
Smallest4.1 GB
QW
SummOrchestra Qwen3 8B GRPO BRLP SAMSUM
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ4_K_M 5.7 GB
MI
MiniCPM5 2B DSpark
MiniCPM · magnitudedev
Params324M
Context128K
Smallest0.7 GB
LF
LFM3-VL-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
VI
VibeThinker 3B
Vibethinker · mradermacher
Params3.09B
Context128K
Best fitQ6_K 3.3 GB
QW
Qwen3.5 9B Claude Fable 5xMerge
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
GL
GLM-ASR-Nano-2512
GLM · Mungert
Params1.59B
Context8K
Best fitBF16 4.2 GB
KU
Kunokukulemonchini 7b
Kunokukulemonchini · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
TO
TooManyMixRolePlay-7B-Story_V3.5
Toomanymixroleplay · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LF
LFM2 1.2B Extract
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB