GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
VO
VoxCPM2
Voxcpm · DennisHuang648
Params912M
Context32K
Best fitF16 4.0 GB
OR
Ornith 1.0 35B Heretic MTP APEX
Ornith · SC117
Params447M
Context—
Smallest0.2 GB
QW
Qwen3 ASR 0.6B
Qwen · unslothai
Params752M
Context64K
Best fitBF16 2.9 GB
LF
LFM2.5 1.2B Instruct
Lfm · bartowski
Params1.17B
Context125K
Best fitBF16 3.9 GB
QW
qwen3.5 4B super coder
Qwen · jica98
Params4.33B
Context256K
Best fitQ4_0 4.2 GB
TI
TIPO 500M
Tipo · KBlueLeaf
Params508M
Context2K
Best fitF16 2.4 GB
MI
avemio digital mistral 7b Sft pretrain 6000 steps merge
Mistral · RichardErkhov
Params7.24B
Context—
Best fitQ4_0 5.1 GB
SM
smollm2 360m aac watch
SmolLM · dcostenco
Params362M
Context8K
Smallest0.2 GB
MU
multilingual e5 large instruct
Multilingual · Ralriki
Params559M
Context512
Best fitF16 2.4 GB
OR
Ornith 1.5 35B A3B Huihui Sangreal MTP ICE
Ornith · gbuzhf
Params35.5B
Context256K
Smallest0.2 GB
LF
LFM2.5 VL 450M
Lfm · LiquidAI
Params354M
Context125K
Best fitF32 2.5 GB
QW
Qwen3 4B RPG Roleplay V2
Qwen · Chun121
Params4B
Context32K
Best fitQ8_0 6.0 GB
BI
bitnet b1.58 2B 4T
Bitnet · microsoft
Params2.41B
Context4K
Smallest1.2 GB
QW
Qwen3.8 Flash Next Uncensored
Qwen · AtomicChat
Params80M
Context256K
Q4_K_M93.9 GB
TI
TinyMistral 248M v2 Instruct
Tinymistral · M4-ai
Params248M
Context32K
Best fitQ8_0 0.9 GB
DA
DanTagGen delta rev2
Dantaggen · KBlueLeaf
Params393M
Context2K
Best fitF16 2.1 GB
LA
Laya
Laya · ggml-org
Params421M
Context8K
Best fitBF16 2.4 GB
QW
XORTRON RICO v3
Qwen · darkc0de
Params27.3B
Context256K
Q4_K_M16.8 GB
SP
Spark X2.6 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Best fitQ6_K 5.1 GB
QW
Qwen3 0.6b Abliterated
Qwen · mradermacher
Params596M
Context40K
Best fitQ5_K_S 1.9 GB
GL
Models Zai Org GLM 4.6V Flash
GLM · MaziyarPanahi
Params9.4B
Context128K
Best fitQ3_K_L 6.0 GB
MI
mistralai_Ministral 3 3B Instruct 2512 BF16
Mistral · lmstudio-community
Params3.43B
Context256K
Best fitQ8_0 5.0 GB
NE
NeoHorse 1 4B
Neohorse · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
MU
Muse Glimmer 30B
Muse · bartowski
Params27.9B
Context128K
Best fitQ8_0 3.7 GB
OR
Ornith 1.5 9B Heretic v2
Ornith · Thunder13240
Params8.95B
Context256K
Best fitIQ3_XS 5.8 GB
JI
JiRackUltra 1b
Jirackultra · CMSManhattan
Params1.78B
Context128K
Q4_K_M1.1 GB
MI
Devstral Small 2 24B Instruct 2512
Mistral · bartowski
Params23.6B
Context384K
Q4_K_M14.3 GB
QW
Qwen3 VL 2B Instruct
Qwen · ggml-org
Params1.72B
Context256K
Best fitQ8_0 3.3 GB
MI
MistralRP Noromaid NSFW Mistral 7B
Mistral · Ttimofeyka
Params7.24B
Context32K
Best fitQ4_0 5.7 GB
QW
Qwen3.8 27B Uncensored
Qwen · BnhHo
Params27.3B
Context256K
Best fitQ8_0 5.8 GB