GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
BE
Bespoke-Nimble-9B-v3
Bespoke · ggml-org
Params8.95B
Context256K
Best fitQ4_K_M 7.9 GB
QW
Models Qwen Qwen3 4B Thinking 2507
Qwen · MaziyarPanahi
Params4B
Context256K
Best fitQ6_K 5.0 GB
QW
Abliterated
Qwen · criscar22
Params7.6B
Context32K
Best fitQ6_K 7.2 GB
NE
NeoHorse-1-4B
Neohorse · ngquocvinh
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
TI
Tinylaya For Testing
Tinylaya · ggml-org
Params90M
Context8K
Best fitQ8_0 0.7 GB
QW
Qwen3.8 27B Pi
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
SM
SmolVLM2 500M Video Instruct
Smolvlm · ggml-org
Params409M
Context8K
Best fitF16 1.7 GB
LF
Lfm2.5-8B-A1B
Lfm · unsloth
Params8.47B
Context125K
Best fitMXFP4_MOE 7.5 GB
SM
SmolVLM 256M Instruct
Smolvlm · ggml-org
Params163M
Context8K
Best fitF16 1.0 GB
LF
Lfm2.5-Vl-3B
Lfm · unsloth
Params2.7B
Context125K
Best fitBF16 7.9 GB
OR
Ornith 1.5 122B UNCENSORED
Ornith · dealignai
Params125B
Context256K
Best fitQ8_0 2.3 GB
QW
Abliterated MiMo V2.6 Distill Qwen 9B
Qwen · BoldingBuilds
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
QW
Qwen3 VL 8B GLM 4.7 Flash Heretic Uncensored Thinking
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · bartowski
Params7.62B
Context4K
Best fitQ6_K_L 7.5 GB
MI
Ministral 3 3B Instruct 2512
Mistral · bartowski
Params3.43B
Context256K
Best fitQ8_0 5.0 GB
QW
Qwen3.5-4B-3D-Coder
Qwen · ccore
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
QW
Qwen3.5 122B A10B
Qwen · bartowski
Params125B
Context256K
Q4_K_M77.6 GB
CL
Clef Flash Heretic
Clef · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
LL
Llama Joycaption Beta One Hf Llava
Llama · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
GE
gemma 4 E2B it uncensored
Gemma · TrevorJS
Params4.65B
Context128K
Best fitQ8_0 6.1 GB
LL
dolphin 2.9 llama3 8b
Llama · bartowski
Params8B
Context8K
Best fitQ5_K_L 7.6 GB
QW
Qwen3 0.6B Abliterated
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
QW
Qwen3 0.6B Heretic Abliterated Uncensored
Qwen · mradermacher
Params596M
Context40K
Best fitQ5_K_S 1.9 GB
FL
FLUX 3 Image Klein
Flux · AtomicChat
Params5.55B
Context256K
Best fitQ8_0 7.7 GB
MI
Ministral-3-8B-Instruct-2512
Mistral · unsloth
Params8.49B
Context256K
Best fitQ5_K_M 7.7 GB
CE
MN 12B Celeste V1.9
Celeste · nothingiisreal
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
QW
Qwen3 0.6B Heretic
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
VI
VieNeu TTS v2
Vieneu · pnnbao-ump
Params294M
Context4K
Smallest0.2 GB
SM
SmolLM2 360M Instruct
SmolLM · HuggingFaceTB
Params362M
Context8K
Best fitQ8_0 1.2 GB
LL
Hermes 3 Llama 3.2 3b Base Fft Chatml Rerun 8xh100 E4 2024 11 24 4k Steps
Llama · bartowski
Params3.21B
Context128K
Best fitF16 7.9 GB