GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Ternary-Bonsai-2-4B
Qwen · prism-ml
Params782M
Context256K
Best fitBF16 3.1 GB
DO
Dolphin3 Cyber 8B
Dolphin · RavichandranJ
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ6_K_L 7.5 GB
QW
Models Qwen Qwen3 8B
Qwen · MaziyarPanahi
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
QW
Huihui Qwen3 14B Abliterated v2
Qwen · mradermacher
Params14.8B
Context40K
Best fitQ2_K 7.6 GB
TR
Translategemma 4b It
Translategemma · bullerwins
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
GE
Gemma-3-12B-It
Gemma · unsloth
Params11.8B
Context128K
Best fitIQ2_XXS 7.3 GB
KI
kitten tts 2
Kitten · KittenML
Params1.73B
Context40K
Smallest1.0 GB
LL
Llama 2 7B Chat
Llama · TheBloke
Params6.74B
Context4K
Best fitQ3_K_S 7.7 GB
BG
bge m3
BGE · gpustack
Params567M
Context8K
Q4_K_M0.4 GB
ST
L3-8B-Stheno-v3.2
Stheno · bartowski
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
QW
Qwen3.8-Flash-Next
Qwen · ggml-org
Params177B
Context256K
Best fitQ8_0 4.8 GB
LL
Meta Llama 3 8B Instruct
Llama · MaziyarPanahi
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
QW
Huihui Qwen3 8B Abliterated v2
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
GE
Gemma 3 4b It
Gemma · lmstudio-community
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
ST
stories15M MOE
Stories · ggml-org
Params8M
Context256
Smallest0.0 GB
QW
Qwen3 Embedding 4B
Qwen · Qwen
Params4B
Context40K
Best fitQ8_0 6.0 GB
NE
NVIDIA Nemotron 3.5 Bolt 74B A9B
Nemotron · unsloth
Params74B
Context256K
Smallest1.0 GB
FA
Jiqing tiny random falcon
Falcon · RichardErkhov
Params—
Context—
Best fitIQ3_M 0.5 GB
QW
Qwen3.5 0.8B
Qwen · ggml-org
Params773M
Context256K
Best fitBF16 2.5 GB
HU
Cyber Tiel Coder 35B A3B
Huihui · peculiar-ragdoll
Params34.7B
Context256K
Q4_K_M22.1 GB
FR
FrontTerrain-HARP-1B
Frontterrain · frontterrain
Params123M
Context512
Best fitF16 0.8 GB
GE
Winnow 12B
Gemma · EldanRing
Params423M
Context256K
Best fitBF16 2.4 GB
QW
Qwen_Qwen3 VL 8B Instruct
Qwen · lmstudio-community
Params8.19B
Context256K
Best fitQ4_K_M 6.7 GB
IN
Index Translate 9B
Index · IndexTeam
Params9.2B
Context256K
Best fitQ4_K_M 7.4 GB
SM
SmolLM2 135M Instruct
SmolLM · bartowski
Params135M
Context8K
Best fitF16 1.0 GB
QW
Huihui Qwythos 9B Claude Mythos 5 1M Abliterated MTP
Qwythos · huihui-ai
Params9.2B
Context1024K
Best fitQ4_K 7.4 GB
WA
Wald-4B v1.2
Wald · org2ai
Params4.2B
Context256K
Best fitQ8_0 6.1 GB
GE
Gemma-3N-E2B-It
Gemma · unsloth
Params4.46B
Context32K
Best fitQ8_0 5.8 GB
QW
Qwen3.8 27B pi
Qwen · bytkim
Params27B
Context256K
Best fitBF16 6.6 GB