GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Best fitQ8_0 6.0 GB
GE
Gemma SEA LION v4 4B VL
Gemma · aisingapore
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Best fitQ8_0 6.5 GB
MI
MinerU2.5 Pro 2605 1.2B
Mineru · mradermacher
Params494M
Context8K
Best fitF16 1.6 GB
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
FL
Flan T5 Xxl
Flan · easygoing0114
Params4.76B
Context512
Best fitQ6_K 7.6 GB
CL
Clef Flash
Clef · taurusduan
Params9.1B
Context256K
Best fitQ4_K_M 7.6 GB
LF
LFM2.5 1.2B Instruct DSpark Draft v1
Lfm · LiquidAI
Params296M
Context125K
Best fitF16 1.2 GB
LL
llama 2 7B Guanaco QLoRA
Llama · TheBloke
Params6.74B
Context2K
Best fitQ3_K_S 7.7 GB
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ4_1 7.6 GB
MI
Mixtral AI CyberTron Ultra
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ6_K 7.2 GB
OR
Ornith 1.5 397B
Ornith · bartowski
Params397B
Context256K
Q4_K_M241.8 GB
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Best fitQ8_0 7.5 GB
HE
HelpingAI-15B
Helpingai · mradermacher
Params15.3B
Context4K
Best fitQ3_K_S 8.0 GB
OR
Orca 2 7B
Orca · TheBloke
Params6.74B
Context4K
Best fitQ3_K_S 7.7 GB
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Best fitQ6_K 7.6 GB
PH
Athene Phi 3.5 mini instruct orpo
Phi · bartowski
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Best fitBF16 7.6 GB
LI
Ling 3.0 Tiny Uncensored Abliterated
Ling · mradermacher
Params7.9B
Context128K
Best fitQ2_K 7.9 GB
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Best fitQ4_K_L 7.9 GB
NE
NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash
Nemotron · ggml-org
Params7.52B
Context128K
Smallest0.6 GB
LL
Llama Song Stream 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Best fitF16 7.9 GB
DE
Janus Pro 1B
DeepSeek · mradermacher
Params1.65B
Context16K
Best fitF16 5.4 GB
TR
TranslatePsy AfriSLM 2B
Translatepsy · mradermacher
Params2.39B
Context256K
Best fitQ6_K 2.9 GB
BI
BioMistral 7B
Biomistral · MaziyarPanahi
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
EX
EXAONE 3.5 7.8B Instruct
Exaone · LGAI-EXAONE
Params7.8B
Context32K
Best fitQ6_K 8.0 GB
OL
AMD OLMo 1B SFT DPO
OLMo · lmstudio-community
Params1.18B
Context2K
Best fitQ8_0 2.8 GB
QW
Qwen3.5 9B
Qwen · Smoffyy
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB