Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
IC
IceTea21EnergyDrinkRPV13 DPOv3 i1
Icetea · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen Image 2.1 ComfyUI
Qwen · vantagewithai
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
LL
gguf lora llama3 midjourney prompt generator
Llama · rvian
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
RU
RusMind.Вспышка 3.5
Rusmind · DICPlay
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
QW
Qwepus
Qwepus · mradermacher
Params8.95B
Context1024K
Q4_K_M5.6 GB
Coding
QW
Qwen3 Reranker 8B GGUF llama cpp
Qwen · Voodisss
Params7.57B
Context40K
Q4_K_M4.7 GB
Embedding
DA
daybreak kunoichi dpo 7b
Daybreak · mradermacher
Params7.24B
Context16K
Q4_K_M4.5 GB
GGUF
LL
Na0s Llama 3.1 8b Pruned 4 Layers
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
GGUF
RE
ReFeed 8B
Refeed · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
Gemma 3 4b it MAX HORROR Uncensored DBL X Imatrix
Gemma · DavidAU
Params4.26B
Context128K
Q4_K_M3.5 GB
Roleplay
GR
granite-8b-code-instruct
Granite · ibm-granite
Params8.05B
Context4K
Q4_K_M4.9 GB
CodingTool use
PH
Phi 3 mini 128k instruct imatrix
Phi · duyntnet
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
llava v1.6 mistral 7b
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Vision
MI
Ministral-3-8B-Reasoning-2512
Mistral · Mungert
Params8.5B
Context256K
Q4_K_M5.5 GB
Reasoning
SP
Spark X2.5 4B
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
GGUF
LF
Lfm2-8B-A1B
Lfm · unsloth
Params8.34B
Context125K
Q4_K_M5.0 GB
GGUF
QW
Qwen3.5 9B Claude Code
Qwen · empero-ai
Params8.95B
Context256K
Q4_K_M5.6 GB
Coding
MI
Young Children Storyteller Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HO
K2-Horizon-3.7B
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
GGUF
PH
Phi 3.5 Mini 3.8B ArliAI RPMax v1.1
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama 3.1 SuperNova Lite
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
godot gemma 4 e4b it
Gemma · bekoozkan
Params7.5B
Context128K
Q4_K_M5.3 GB
CodingVision
PH
Phi 4 Mini Reasoning
Phi · bartowski
Params3.84B
Context128K
Q4_K_M2.5 GB
CodingReasoning
MI
akjindal53244_arithmo-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Reasoning
LL
Meta Llama 3.1 8B Claude
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.5 9B Kimi K3 Distilled
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
Reasoning
JA
Jais-3-3.7B
Jais · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
Reasoning
AI
Aiaudiotranslate
Aiaudiotranslate · Anthropic-ai
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MI
teknium_airoboros-mistral2.2-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LO
Loopl 4b
Loopl · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Tool useVision

Other sizes

Keep browsing