GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
MI
MiniCPM V 4.7 1B LocalAI
MiniCPM · weareapexcreators
Params1.01B
Context256K
Best fitQ8_0 2.0 GB
NA
vllm-d3-nano
Nano · ggml-org
Params2.4B
Context256K
Best fitBF16 5.7 GB
QW
Qwen2.5 VL 7B NSFW Caption V3
Qwen · bartowski
Params7.62B
Context125K
Best fitQ6_K_L 7.5 GB
PH
Phi 3 mini 4k instruct GGUF Imatrix smashed
Phi · PrunaAI
Params3.82B
Context4K
Best fitQ8_0 7.8 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · richardyoung
Params7.62B
Context32K
Best fitQ5_K_M 6.4 GB
OR
Ornith 1.5 122B
Ornith · bartowski
Params125B
Context256K
Q4_K_M78.9 GB
LL
Lexi-Llama-3-8B-Uncensored
Llama · bartowski
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
QW
Qwopus3.5 9B Coder Templatefix Hf
Qwopus · Jackrong
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
ST
StartLux Decision v1 4B Q8 0
Startlux · startlux-models
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
LI
LightOnOCR 3 4B
Lightonocr · prithivMLmods
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
HA
Harrier Oss v1 0.6b
Harrier · mradermacher
Params596M
Context32K
Best fitF16 2.6 GB
DE
decider 4b
Decider · Mapika
Params4.2B
Context256K
Best fitQ8_0 6.1 GB
QW
Qwen3.8 9B Distill Uncensored Heretic
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ4_1 7.6 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · noctrex
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB
QW
Qwen3-Vl-2B-Thinking
Qwen · unsloth
Params1.72B
Context256K
Best fitBF16 4.9 GB
TR
Translategemma 4b It
Translategemma · SandLogicTechnologies
Params3.88B
Context128K
Best fitQ5_K_M 4.5 GB
RO
TheDrummer Rocinante XL 16B v1
Rocinante · bartowski
Params16.1B
Context128K
Q4_K_M10.1 GB
QW
Qwen3.5 4B heretic imatrix
Qwen · FadedRedStar
Params334M
Context256K
Best fitQ5_K_M 4.6 GB
QW
Qwen3.8 27B abliterated MTP
Qwen · hotdogs
Params27B
Context256K
Q4_K_M16.8 GB
QW
Qwen2.5 VL 3B Instruct
Qwen · mradermacher
Params3.09B
Context125K
Best fitF16 7.0 GB
XX
encoder
Xxl · chatpig
Params219M
Context512
Best fitQ6_K 7.6 GB
OR
Ornith 1.5 9b Mtp
Ornith · protoLabsAI
Params456M
Context256K
Best fitQ4_K_M 7.4 GB
LY
Lythri 4B A2B
Lythri · bartowski
Params4.63B
Context128K
Best fitQ8_0 6.0 GB
NE
NeoHorse-1-4B
Neohorse · TokenRhythm
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
DE
WhiteRabbitNeo WhiteRabbitNeo V3 7B
Deephat · bartowski
Params7.62B
Context32K
Best fitQ6_K_L 7.5 GB
LF
LiquidAI_LFM2.5 1.2B Instruct
Lfm · lmstudio-community
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
QW
Qwen3 VL 8B Heretic Stable
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB
GL
Glm-4-9B-0414
GLM · unsloth
Params9.4B
Context32K
Best fitQ5_K_XL 7.9 GB
GR
Granite 20b Code Instruct
Granite · bartowski
Params20B
Context8K
Best fitIQ2_M 7.8 GB
FO
Foundation Sec 8B Reasoning
Foundation · fdtn-ai
Params8B
Context128K
Best fitQ4_K_M 6.5 GB