GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
AC
xl_regrind_v1
Acestep · mdmachine
Params4.99B
Context32K
Best fitQ8_0 6.9 GB
QW
Qwen3.6 35B A3B DFlash
Qwen · jcuypers
Params474M
Context256K
Best fitQ4_K_M 0.9 GB
MA
madlad400 3b mt
Madlad · google
Params2.94B
Context—
Smallest1.0 GB
FB
fblgit juanako 7b UNA
Fblgit · RichardErkhov
Params7.24B
Context—
Best fitQ4_0 5.1 GB
QW
Qwen3 4b Z Image Turbo AbliteratedV1
Qwen · BennyDaBall
Params4B
Context40K
Best fitQ8_0 6.0 GB
QW
Qwen3.8-27B-RANA-abliterated
Qwen · abliterant
Params27B
Context256K
Best fitQ8_0 5.8 GB
QW
Qwen3.8 9B heretic uncensored NVFP4
Qwen · Noobito45
Params8.95B
Context256K
Best fitQ8_0 7.6 GB
MI
mistralai_mistral-7b-instruct-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
MI
MiniCPM V 4.7 35B A3B CRACKv5 2p0
MiniCPM · dealignai
Params34.7B
Context256K
Best fitBF16 2.0 GB
DE
DeepSeek R1 Distill Qwen 1.5B
DeepSeek · bartowski
Params1.78B
Context128K
Best fitF32 7.8 GB
QW
Qwen3 8B
Qwen · bartowski
Params8.19B
Context32K
Best fitQ5_K_L 7.9 GB
LF
LFM2.5 2.6B GGUF shards
Lfm · Arki05
Params2.7B
Context128K
Best fitBF16 7.9 GB
QW
Qwen3.5 9B Abliterated
Qwen · Abiray
Params8.95B
Context256K
Best fitQ5_K_S 7.9 GB
OR
Ornith 1.5 9B UNCENSORED
Ornith · dealignai
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
TE
HY MT1.5 1.8B
Tencent · tencent
Params1.79B
Context256K
Best fitQ8_0 2.9 GB
GE
gemma 2b fine tune
Gemma · arnavj007
Params2B
Context—
Best fitF16 0.5 GB
PH
Phi 3.5 Mini Instruct
Phi · bartowski
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
OR
OrcaCyber Nano 1.0
Orcacyber · orcarouter
Params1.92B
Context256K
Best fitQ8_0 2.8 GB
LL
Dolphin 3.0 Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Best fitQ5_K_L 7.6 GB
ME
Medgemma-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Best fitQ8_K_XL 6.8 GB
QW
Qwen3 14B Abliterated
Qwen · bartowski
Params14.8B
Context40K
Best fitIQ3_XXS 7.8 GB
CY
z image cyberrealistic v9
Cyberrealistic · gimly13
Params6.15B
Context40K
Best fitQ6_K 7.8 GB
TI
py007_tinyllama-1.1b-chat-v0.3
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
FL
FLUX.2 klein 9B
Flux · leejet
Params9.08B
Context—
Best fitQ4_0 6.8 GB
QW
Qwen2.5-VL-3B-Instruct
Qwen · Mungert
Params3.1B
Context125K
Best fitBF16 7.0 GB
EM
Embeddinggemma 300m Qat Q8_0 Unquantized
Embeddinggemma · ggml-org
Params303M
Context2K
Best fitQ8_0 1.0 GB
QW
Qwen3VL 8B Uncensored HauhauCS Aggressive
Qwen · HauhauCS
Params8.2B
Context256K
Best fitQ4_K_M 6.7 GB
GE
Gemma 2 2b It
Gemma · bartowski
Params2.61B
Context8K
Best fitQ8_0 4.2 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
QW
Qwen2.5 Coder 0.5B Instruct AWQ
Qwen · Qwen
Params630M
Context32K
Q4_K_M0.5 GB