GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
LL
ultravox v0 5 llama 3 2 1b
Llama · ggml-org
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
SP
Spark X2.5 4B Abliterated
Spark · SC117
Params4.11B
Context1024K
Best fitQ6_K 5.1 GB
MI
maywell Mistral ko 7B v0.1
Mistral · RichardErkhov
Params1.5B
Context—
Best fitIQ3_XS 4.0 GB
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
TI
TIPO 200M Ft
Tipo · KBlueLeaf
Params203M
Context2K
Best fitF16 1.4 GB
SA
Sage A1.5 Heretic Abliterated Uncensored
Sage · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
LF
LFM2 350M
Lfm · LiquidAI
Params354M
Context125K
Best fitF16 1.7 GB
QW
Qwopus3.5 4B Coder MTP HF
Qwen · Jackrong
Params334M
Context256K
Best fitQ6_K 5.2 GB
QW
Qwen3 235B A22B Instruct 2507
Qwen · bartowski
Params235B
Context256K
Q4_K_M142.6 GB
QW
Dongfangshuo Qwen3.8 27B NSFW Novel
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
HO
Holo4 35B A3B
Holo · bartowski
Params34.7B
Context256K
Q4_K_M22.3 GB
JI
Hf Jinen v2 Small
Jinen · togatogah
Params110M
Context1K
Best fitF16 1.0 GB
MI
Source Hf Minicpm V 4.7 1b
MiniCPM · ngquocvinh
Params1.01B
Context256K
Best fitQ8_0 2.0 GB
QW
Qwen2.5 VL 7B Abliterated Caption It
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
QW
Qwen3.5 4B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
LL
Llama 3.1 8B Lexi Uncensored
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Andy 4.2
Qwen · Mindcraft-CE
Params8.95B
Context256K
Best fitQ2_K 5.2 GB
MI
ehartford_dolphin-2.2.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Qwen3 4B Abliterated
Qwen · mradermacher
Params4B
Context40K
Best fitQ8_0 6.0 GB
MI
MiMo V2.6 Flash RL NVFP4
Mimo · ProCreations
Params310B
Context1024K
Best fitBF16 4.9 GB
MI
MiniCPM V 4.7 35B A3B Abliterated
MiniCPM · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
DE
DeepSeek R1 0528 Qwen3 8B
DeepSeek · bartowski
Params8.19B
Context128K
Best fitQ3_K_M 5.8 GB
SP
rizzo flow
Spark · rizzoaiacademy
Params1.71B
Context1024K
Best fitBF16 5.1 GB
CO
Yi Coder 9B
Coder · RichardErkhov
Params8.83B
Context128K
Best fitQ3_K_L 6.0 GB
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
QW
Qwen3.5 27B Abliterated
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
DO
Dots3 Note Prev
Dots · bartowski
Params280B
Context512K
Q4_K_M171.1 GB
LL
Llama 3.2 1B Instruct
Llama · hugging-quants
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
DE
DeepHat V1 7B
Deephat · mradermacher
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
RO
RoMistral-7b-Instruct
Romistral · legraphista
Params7.24B
Context32K
Best fitQ4_K 5.9 GB