Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
LA
law chat
Law · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
JA
JapanRailwaysimLLM
Japanrailwaysimllm · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Amory0201
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
HU
Huihui4 8B A4B v2
Huihui · mradermacher
Params8.1B
Context256K
Q4_K_M5.4 GB
GGUF
LL
Llama 3.3 8B Instruct
Llama · Green-Eye
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3-Reranker-4B
Qwen · gorse-io
Params4.02B
Context40K
Q4_K_M2.5 GB
EmbeddingReasoningTool use
LL
llama-3-Nephilim-v2-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 mini 4k instruct imatrix
Phi · duyntnet
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
HA
Hathor v4.6 i1
Hathor · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AU
AutoCoder S 6.7B
Autocoder · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama-3-Oasis-v1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
CO
Cosmos Reason1 7B Heretic
Cosmos · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 Youko 8b Instruct
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
GE
Gemma4 E4B Mahou Nsfw
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
ReasoningRoleplayTool use
PY
Pythia 6.9b Deduped
Pythia · mradermacher
Params6.86B
Context2K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B Cyberpunk2077 Xiaotian All4 5pass
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
OP
Openhands Lm 7b v0.1
Openhands · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
DE
Deepseek Llm 7b Chat Abliterated
DeepSeek · mradermacher
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
IG
Ignis 7B DPO Laser
Ignis · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
NE
Daredevil-8B-abliterated-dpomix
Neuraldaredevil · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
AM
amalia-desenrasca-9b
Amalia · mradermacher
Params9.15B
Context32K
Q4_K_M5.6 GB
Tool use
LL
Llama-2-7b-pruned70-retrained-gsm8k
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
CodingVision
ME
MedMO 8B Next
Medmo · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Reasoning
QW
Qwen2.5 Math 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context4K
Q4_K_M4.7 GB
GGUF

Other sizes

Keep browsing