GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
PI
piano-medley-7b
Piano · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
AM
AMALIA 9B 0626 SFT
Amalia · NoxNotreve
Params9.15B
Context32K
Best fitQ4_K_M 7.5 GB
RW
rwitz_go-bruins-v2
Rwitz · TheBloke
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
FR
FrogNano 4B 2609
Frognano · NANI-Nithin
Params4.33B
Context256K
Best fitQ8_0 6.2 GB
JA
Jais-3-MoVA-36B-A4B
Jais · mradermacher
Params37.4B
Context512K
Q4_K_M22.4 GB
LL
Colossal LLaMA 2 7b Base
Llama · mradermacher
Params7.04B
Context4K
Best fitIQ3_S 7.9 GB
MI
mathstral-7B-v0.1
Mistral · bartowski
Params7.25B
Context32K
Best fitQ6_K_L 7.6 GB
AN
Anubis Mini 8B v1
Anubis · bartowski
Params8B
Context128K
Best fitQ5_K_L 7.8 GB
NO
nomic-embed-text-v1.5
Nomic · second-state
Params137M
Context2K
Best fitF16 1.1 GB
GE
Gemma-3-270M-It-Qat
Gemma · unsloth
Params268M
Context32K
Best fitF16 1.2 GB
JI
Jina Embeddings v5 Text Nano Classification
Jina · jinaai
Params212M
Context8K
Best fitF16 1.2 GB
NO
Novel Humanize 4b
Novel · bartowski
Params4.02B
Context40K
Best fitQ8_0 6.0 GB
NI
Ninja-v1-NSFW-RP
Ninja · mradermacher
Params7.24B
Context4K
Best fitQ6_K 7.5 GB
LL
dolphin-2.9.1-llama-3-8b
Llama · legraphista
Params8B
Context8K
Best fitQ5_K 7.3 GB
HU
HuatuoGPT O1 7B
Huatuogpt · mradermacher
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
HU
Huihui LFM2.5 8B A1B Abliterated
Huihui · mradermacher
Params8.47B
Context125K
Best fitQ5_K_S 8.0 GB
LF
LFM2.5-24B-A2B
Lfm · ngquocvinh
Params24B
Context128K
Best fitQ1_0 6.8 GB
QW
Darrellbest Qwen Image 2.1 PE T2I Heretic
Qwen · darrellbest
Params8.95B
Context256K
Best fitQ4_K_M 7.2 GB
LL
georgesung_llama2_7b_chat_uncensored
Llama · TheBloke
Params6.74B
Context2K
Best fitQ3_K_S 7.7 GB
TO
tolkio models
Tolkio · datacrunch
Params1.94B
Context256K
Smallest0.6 GB
NO
Nora0.1 1.7B
Nora · mradermacher
Params1.72B
Context40K
Best fitQ6_K 2.9 GB
JI
Jina Embeddings v5 Text Small Clustering
Jina · jinaai
Params596M
Context32K
Best fitF16 2.6 GB
LF
LFM3 1.7B Heretic Abliterated
Lfm · Vortecks
Params1.73B
Context256K
Best fitBF16 5.6 GB
NE
NeuralDaredevil 8B abliterated
Neuraldaredevil · QuantFactory
Params8B
Context8K
Best fitQ5_1 7.6 GB
WH
whisper large v3 turbo
Whisper · oxide-lab
Params809M
Context—
Best fitQ8_0 1.5 GB
GL
Glm-4.1V-9B-Thinking
GLM · unsloth
Params9.4B
Context64K
Best fitQ5_K_M 7.9 GB
MI
Mistral NeMo 12B Abliterated
Mistral · mradermacher
Params12.2B
Context128K
Best fitQ3_K_M 7.9 GB
DE
DeepSeek R1 Distill Llama 8B Abliterated
DeepSeek · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
QW
Qwen3 Omni 30B A3B Captioner
Qwen · mradermacher
Params30.5B
Context64K
Best fitIQ1_S 7.7 GB
PH
phi 2 electrical engineering
Phi · TheBloke
Params2.78B
Context2K
Best fitQ8_0 6.1 GB