GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.8-27B-DFlash2
Qwen · incoai
Params1.92B
Context256K
Best fitBF16 4.5 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
BG
bge m3 Q8 0
BGE · ggml-org
Params567M
Context8K
Best fitQ8_0 1.9 GB
LF
LFM3 Audio 2B
Lfm · LiquidAI
Params1.73B
Context125K
Best fitBF16 5.6 GB
QW
Qwen3.5 Text 9B NSFW RP RolePlay
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
QW
Qwen2.5 Coder 3B Instruct
Qwen · bartowski
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
GL
Glm 4 9b Chat
GLM · QuantFactory
Params9.4B
Context128K
Best fitQ3_K_M 5.9 GB
IN
Index Translate 2B
Index · IndexTeam
Params1.94B
Context256K
Best fitF16 4.8 GB
MI
MiniCPM V 4.7 35B A3B DSpark
MiniCPM · openbmb
Params389M
Context256K
Best fitF16 1.4 GB
NE
NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB LiLiCorr
Nemotron · nvidia
Params1.11B
Context256K
Smallest1.0 GB
QW
DFlash
Qwen · Alittlehammmer
Params1.73B
Context256K
Best fitBF16 4.1 GB
GE
Gemma 3 1B It GLM 4.7 Flash Heretic Uncensored Thinking
Gemma · Andycurrent
Params1B
Context32K
Best fitF16 2.7 GB
QW
Swift 1.5 Qwen3.8 27b
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
QW
Qwen3 0.6B Heretic Abliterated Uncensored
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
QW
Qwen3.5 9b Nsfw Captioning v5
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
NE
Models Nvidia Nemotron Orchestrator 8B
Nemotron · MaziyarPanahi
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
QW
Qwen3 0.6B Heretic Heresy Edition
Qwen · mradermacher
Params596M
Context40K
Best fitQ5_K_S 1.9 GB
SM
SmolLM2 360M Instruct
SmolLM · bartowski
Params362M
Context8K
Best fitF16 1.6 GB
DI
Dirty Muse Writer v01 Uncensored Erotica NSFW i1
Dirty · mradermacher
Params9.24B
Context8K
Best fitIQ1_M 5.9 GB
LL
Impish_LLAMA_4B_V2
Llama · SicariusSicariiStuff
Params4.5B
Context128K
Best fitQ6_K 5.3 GB
QW
Models Qwen Qwen3 4B Thinking 2507
Qwen · MaziyarPanahi
Params4B
Context256K
Best fitQ6_K 5.0 GB
QW
Abliterated
Qwen · criscar22
Params7.6B
Context32K
Best fitQ4_K_M 5.7 GB
NE
NeoHorse-1-4B
Neohorse · ngquocvinh
Params4.21B
Context256K
Best fitQ6_K 5.8 GB
TI
Tinylaya For Testing
Tinylaya · ggml-org
Params90M
Context8K
Best fitQ8_0 0.7 GB
QW
Qwen3.8 27B Pi
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
SM
SmolVLM2 500M Video Instruct
Smolvlm · ggml-org
Params409M
Context8K
Best fitF16 1.7 GB
LF
Lfm2.5-8B-A1B
Lfm · unsloth
Params8.47B
Context125K
Best fitIQ3_S 5.7 GB
SM
SmolVLM 256M Instruct
Smolvlm · ggml-org
Params163M
Context8K
Best fitF16 1.0 GB
LF
Lfm2.5-Vl-3B
Lfm · unsloth
Params2.7B
Context125K
Best fitQ8_K_XL 6.0 GB
OR
Ornith 1.5 122B UNCENSORED
Ornith · dealignai
Params125B
Context256K
Best fitQ8_0 2.3 GB