GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen2.5 7B Instruct Heretic
Qwen · saidutta69
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
PY
Pythia 1b
Pythia · mradermacher
Params1.01B
Context2K
Best fitQ6_K 2.4 GB
LL
dolphin 2.9 llama3 8b
Llama · QuantFactory
Params8B
Context8K
Best fitQ5_1 7.6 GB
PR
Privacy Filter
Privacy · LocalAI-io
Params1.4B
Context128K
Best fitF16 3.4 GB
GE
Gemma-4-E4B-It
Gemma · ente-ai
Params7.5B
Context128K
Best fitQ5_K_M 7.4 GB
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Best fitQ5_K_M 7.6 GB
MI
MiniCPM5 2B DSpark
MiniCPM · openbmb
Params324M
Context128K
Smallest0.7 GB
QW
Qwen3.6 35B A3B Uncensored Heretic Native MTP Preserved
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
SP
Spark X2.6 4B Uncensored GSQ RCO
Spark · abenzerps
Params4.11B
Context1024K
Best fitIQ4_XS 4.3 GB
QW
Qwen3.8 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
GE
Gemma 4 E4B It Ultra Uncensored Heretic
Gemma · mradermacher
Params7.52B
Context128K
Best fitQ5_K_M 7.7 GB
GR
granite 20b functioncalling i1
Granite · mradermacher
Params20B
Context8K
Best fitIQ2_M 7.8 GB
PY
Pythia 410m Deduped
Pythia · mradermacher
Params405M
Context2K
Best fitQ4_0 1.6 GB
QW
Qwen3.8 2B Distill Heretic
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ6_K 2.5 GB
MI
Voxtral Small 24B 2507
Mistral · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
AU
Audar ASR V1 Turbo
Audar · audarai
Params2.03B
Context64K
Q4_K_M1.3 GB
QW
Qwen3 8B Socratic SFT
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
QW
Crow 9B HERETIC 4.6
Qwen · Crownelius
Params8.95B
Context256K
Best fitQ4_K_M 7.2 GB
GE
Skaldic Gemma 4 E4B
Gemma · mradermacher
Params7.46B
Context128K
Best fitQ5_K_S 7.6 GB
QW
Qwen2-1.5B-Instruct
Qwen · legraphista
Params1.54B
Context32K
Best fitBF16 3.8 GB
AI
jondurbin_airoboros-m-7b-3.1
Airoboros · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
JA
Jais-3-7B
Jais · mradermacher
Params9B
Context512K
Best fitQ5_K_S 8.0 GB
QW
Wrench 9B Q4 K M
Qwen · ClankLabs
Params8B
Context256K
Best fitQ4_K_M 7.2 GB
UM
Umievo-itr012-Gleipnir-7B
Umievo · mmnga
Params7.24B
Context8K
Best fitQ6_K 7.5 GB
KI
Kisoku 1.6b Chat
Kisoku · mradermacher
Params1.6B
Context128K
Best fitQ6_K 2.2 GB
HU
Hunyuan 1.8B Instruct
Hunyuan · bartowski
Params1.79B
Context256K
Best fitBF16 4.6 GB
GR
Granite 3b Code Base 128k
Granite · RichardErkhov
Params3.48B
Context125K
Best fitQ8_0 6.9 GB
QW
Qwen2.5 7B Instruct Uncensored
Qwen · WSDW
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GE
Gemma 4 E4B It Apostate
Gemma · mradermacher
Params8.13B
Context128K
Best fitQ5_K_M 7.7 GB
AU
Audar TTS V1 Turbo
Audar · audarai
Params1.64B
Context32K
Best fitQ8_0 2.5 GB