GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
GE
Gemma 3 4b It Qat
Gemma · bartowski
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
LI
LightOnOCR 3 0.8B
Lightonocr · Prathat
Params752M
Context256K
Best fitF16 2.4 GB
DE
DeepSeek R1 Distill Qwen 7B Abliterated v2
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ6_K 7.2 GB
QW
Qwen3.8 27B
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
LL
Llama 3.2 1B Instruct
Llama · DavidAU
Params1.24B
Context128K
Best fitQ6_K 1.8 GB
QW
Qwen3.5 4B Heretic
Qwen · Biomanticus
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
CH
ChemLLM 20B Chat SFT i1
Chemllm · mradermacher
Params19.9B
Context32K
Best fitIQ2_XXS 7.7 GB
TU
Tulu 2 7B Helpful
Tulu · mradermacher
Params6.74B
Context8K
Best fitIQ3_M 7.9 GB
QW
Huihui ai.Qwen3 8B Abliterated
Qwen · DevQuasar
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
LI
LightOnOCR 3 0.8B
Lightonocr · prithivMLmods
Params752M
Context256K
Best fitBF16 2.4 GB
PH
dolphin 2 6 phi 2
Phi · TheBloke
Params2.78B
Context2K
Best fitQ8_0 6.1 GB
OC
Occamy 1.0
Occamy · bartowski
Params34.7B
Context256K
Q4_K_M22.3 GB
QW
Bella Bartender Qwen3.5 2b
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
SM
SmolLM 1.7B
SmolLM · bartowski
Params1.71B
Context2K
Best fitQ8_0 3.9 GB
GL
GLM OCR
GLM · mradermacher
Params891M
Context128K
Best fitF16 2.9 GB
QW
Qwen2.5 VL 32B Instruct Heretic
Qwen · mradermacher
Params32.8B
Context125K
Q4_K_M19.9 GB
NO
normistral 7b warm instruct
Normistral · ltg
Params7.25B
Context2K
Best fitQ6_K 7.5 GB
RE
Reggie v2 7b
Reggie · mradermacher
Params7.62B
Context128K
Best fitQ6_K 7.2 GB
NE
NeoHorse 1 9B Uncensored GSQ RCO
Neohorse · abenzerps
Params8.95B
Context256K
Best fitIQ4_XS 6.3 GB
CO
Yi Coder 9B Chat
Coder · bartowski
Params8.83B
Context128K
Best fitQ5_K_L 7.7 GB
QW
qwen3 tts 0 6b custom voice mix precision
Qwen · Arm
Params83M
Context8K
Best fitQ8_0 1.3 GB
HU
HuatuoGPT O1 7B
Huatuogpt · bartowski
Params7.62B
Context32K
Best fitQ6_K_L 7.5 GB
MI
sciphi_sciphi-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
ZD
TaichuAI/ZDTaichu5.0-9B
Zdtaichu · ngquocvinh
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
QW
Qwen2.5 3B Instruct
Qwen · ThomasBaruzier
Params3.09B
Context32K
Best fitF16 7.0 GB
IN
InternVL3_5 30B A3B
Internvl · bartowski
Params30.5B
Context40K
Q4_K_M18.6 GB
SE
sexeh time testing
Sexeh · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
LL
harborwater_open-llama-3b-v2-wizard-evol-instuct-v2-196k
Llama · TheBloke
Params3.43B
Context2K
Best fitQ6_K 6.9 GB
GE
Gemma 4 E4B xReadable zhTW OCR v1
Gemma · xCloudinfo
Params7.46B
Context128K
Best fitQ4_K_M 7.2 GB
MO
moondream2
Moondream · moondream
Params1.42B
Context2K
Best fitF16 5.0 GB