GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
HE
Hemmingway 1
Hemmingway · botp
Params27.3B
Context256K
Q4_K_M17.4 GB
QW
Qwen1.5-0.5B-Chat-AWQ-fp16
Qwen · Qwen
Params620M
Context32K
Best fitQ8_0 2.0 GB
HU
HunyuanOCR
Hunyuanocr · prithivMLmods
Params539M
Context128K
Best fitF32 3.5 GB
QW
Qwen3 4B
Qwen · lmstudio-community
Params4B
Context32K
Best fitQ8_0 6.0 GB
MA
MN 12B Mag Mell R1
Mag · bartowski
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
HO
K2 Horizon MoVA 36B A4B Uncensored
Horizon · mradermacher
Params37.4B
Context512K
Q4_K_M22.4 GB
OP
openchat-3.6-8b-20240522
Openchat · legraphista
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
OL
allenai_olmOCR 2 7B 1025
OLMo · lmstudio-community
Params7.62B
Context125K
Best fitQ4_K_M 5.7 GB
QW
Qwen3 Sex
Qwen · ystemsrx
Params4B
Context256K
Best fitQ8_0 6.0 GB
QW
Swift Qwen3.8 27b
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
QW
Qwen3.5 4B Claude 4.6 Opus Reasoning Distilled v2
Qwen · Jackrong
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
HU
Hunyuan-MT-7B
Hunyuan · Mungert
Params7.5B
Context256K
Best fitQ4_K_S 6.0 GB
QW
Jev Style Qwen3.5 2B Decision
Qwen · chaoliangUNSW
Params1.94B
Context256K
Best fitBF16 4.8 GB
SM
SmolLM2 135M Instruct
SmolLM · mradermacher
Params135M
Context8K
Best fitQ2_K 0.8 GB
QW
Qwen3.8 27B
Qwen · KikoCis
Params26.9B
Context256K
Q4_K_M16.8 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · lmstudio-community
Params7.25B
Context32K
Best fitQ4_K_M 5.9 GB
MI
MiniCPM V 4.7 1B abliterated heretic
MiniCPM · Abiray
Params1.01B
Context256K
Best fitQ8_0 2.0 GB
QW
Sakura 1.5B Qwen2.5 v1.0 GGUF IMX
Qwen · shing3232
Params1.78B
Context128K
Smallest1.3 GB
LA
m51Lab MiniMax M2.7 REAP 139B A10B
Lab · mradermacher
Params139B
Context192K
Q4_K_M84.2 GB
GE
avemio digital gemma 2b Sft pretrain 6000 steps merge
Gemma · RichardErkhov
Params2.5B
Context8K
Best fitQ4_K 2.3 GB
QW
Qwen2.5 7B Instruct
Qwen · paultimothymooney
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
IN
InternVL3_5 8B
Internvl · bartowski
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
GE
gemma 4 E2B it
Gemma · AtomicChat
Params4.65B
Context128K
Best fitQ6_K 4.9 GB
ON
OneJev 27B
Onejev · bartowski
Params27B
Context256K
Q4_K_M17.2 GB
QW
Qwen3.8 Flash Next Coder cut240
Qwen · Schiltmans
Params56.6B
Context256K
Smallest4.5 GB
TI
TIPO v2.1 1B A200M
Tipo · KBlueLeaf
Params1.01B
Context4K
Best fitF16 2.9 GB
SM
SmolLM2 135M
SmolLM · QuantFactory
Params135M
Context8K
Best fitQ2_K 0.8 GB
MI
MiniCPM5 2B
MiniCPM · bartowski
Params2.52B
Context128K
Best fitBF16 5.9 GB
QW
Qwen2.5 Coder 1.5B Instruct Abliterated
Qwen · bartowski
Params1.78B
Context32K
Best fitF16 4.3 GB
GL
GLM 5.3 Flash UNCENSORED mlx oQ8e DFlash2
GLM · Solstice-AI
Params1.17B
Context1024K
Best fitBF16 3.0 GB