GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwable v1
Qwable · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
MI
leolm_leo-mistral-hessianai-7b-chat
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
BR
Breeze 7B 32k Instruct V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Best fitQ6_K 7.7 GB
ST
Starcoderplus
StarCoder · mradermacher
Params15.8B
Context8K
Best fitIQ3_S 7.8 GB
MO
Morpho 1.5B Distill
Morpho · mradermacher
Params1.54B
Context128K
Best fitQ6_K 2.0 GB
OV
OvisOCR2
Ovisocr · bartowski
Params752M
Context256K
Best fitBF16 2.4 GB
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · mradermacher
Params8B
Context8K
Best fitQ5_K_M 7.3 GB
DE
DeepSeek R1 Distill Qwen 1.5B uncensored
DeepSeek · mradermacher
Params1.78B
Context128K
Best fitF16 4.3 GB
QW
Qwen3 1.7b Abliterated
Qwen · mradermacher
Params1.72B
Context40K
Best fitQ6_K 2.9 GB
LF
LFM2.5 24B A2B Uncensored
Lfm · mradermacher
Params24B
Context128K
Q4_K_M14.5 GB
MI
mistral 7b orpo capybara reproduction
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.7 GB
HY
Hy-Mt2-1.8B
unsloth
Params1.79B
Context256K
Best fitBF16 4.6 GB
AI
airoboros l2 7B gpt4 2.0
Airoboros · TheBloke
Params6.74B
Context4K
Best fitQ3_K_S 7.7 GB
QW
Qwen3-Vl-8B-Instruct-1M
Qwen · unsloth
Params8.19B
Context977K
Best fitQ5_K_M 7.6 GB
EL
Elephant 1
Elephant · prithivMLmods
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
LO
Loopl 0.8b
Loopl · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
FR
Frostwind 10.7B v1
Frostwind · TheBloke
Params10.7B
Context4K
Best fitQ3_K_L 7.8 GB
LF
LFM2 VL 3B
Lfm · LiquidAI
Params2.57B
Context125K
Best fitF16 7.7 GB
MI
cognitivecomputations_dolphin-2.6-mistral-7b-dpo
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
MI
RenCoder Ministral 8B Instruct 2410
Mistral · mradermacher
Params8B
Context32K
Best fitQ5_K_M 7.4 GB
ME
Mephisto 1.0 4B
Mephisto · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
ME
Mephisto 1.5 4B
Mephisto · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
MI
ANITA NEXT 24B Magistral 2506 VISION ITA
Mistral · mradermacher
Params23.6B
Context128K
Best fitIQ1_M 7.6 GB
LF
LFM2.5 8B A1B APEX
Lfm · mudler
Params8.47B
Context125K
Smallest3.6 GB
GE
GemmaTheWriter
Gemma · DavidAU
Params9.24B
Context8K
Best fitQ3_K_S 7.7 GB
IN
Internlm2 Math 7b
InternLM · mradermacher
Params7.74B
Context8K
Best fitQ6_K 7.9 GB
AT
N ATLaS
Atlas · QuantFactory
Params8B
Context128K
Best fitQ5_1 7.6 GB
QW
ThinkingCap Qwen3.8 27B abliterated
Qwen · IstroSec
Params27.3B
Context256K
Q4_K_M17.0 GB
QW
Qwen2.5 Coder 14B E2E Tests
Qwen · mradermacher
Params14.8B
Context32K
Best fitQ2_K 7.9 GB
PY
Pythia 1.4b Deduped
Pythia · mradermacher
Params1.41B
Context2K
Best fitQ6_K 3.3 GB