Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SW
Swallow-7b-plus-hf
Swallow · mradermacher
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Josiefied Qwen3 8B Abliterated v1
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
LI
limstral 7B v0.1
Limstral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
IN
L3.1 Instruct Guru 8B
Instruct · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
AR
ARWKV 7B Preview 0.1
Arwkv · mradermacher
Params8.29B
Context1024K
Q4_K_M5.2 GB
GGUF
QW
QwQ LCoT2 7B Instruct i1
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Reasoning
OM
OmniLing V1 8b experimental i1
Omniling · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GR
granite 4.2 3b
Granite · NANI-Nithin
Params3.66B
Context128K
Q4_K_M2.2 GB
ReasoningTool use
PR
Prism Coder 9b
Prism · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingTool useVision
EU
Eurus 2 7B PRIME
Eurus · bartowski
Params7.62B
Context4K
Q4_K_M4.7 GB
GGUF
MI
Mistral-7B-v0.3
Mistral · legraphista
Params7.25B
Context32K
Smallest1.6 GB
GGUF
MI
Mistral 7B Instruct KhanAcademy v0.2 i1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Roleplay
KU
Kunoichi DPO v2 7B
Kunoichi · brittlewis12
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
llava llama 3 8b v1 1
Llama · xtuner
Params8B
Context8K
Smallest4.9 GB
Vision
LL
MathCoder2 CodeLlama 7B
Llama · bartowski
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
DE
Josiefied DeepSeek R1 0528 Qwen3 8B Abliterated v1
DeepSeek · mradermacher
Params8.19B
Context128K
Q4_K_M5.0 GB
Reasoning
LL
DarkIdol Llama 3.1 8B Instruct 1.2 Uncensored
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
QW
Qwen3 Reranker 4B
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
Embedding
LL
Llama 3 8B Instruct abliterated
Llama · failspy
Params8B
Context8K
Smallest4.9 GB
GGUF
SH
models--automerger--ShadowT3q-7B
Shadow · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PU
L3.1 Purosani i1
Purosani · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
ToolACE 2.5 Llama 3.1 8B Heretic
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
PI
Piaget 8B
Piaget · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
QW
GrayLine Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
QW
Qwen3.5 9B Kimi K3 Distilled
Qwen · somenetworktrouble
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
AL
AllOverButTheCrying-7B-slerp
Alloverbutthecrying · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
RedSage Qwen3 8B Ins
Qwen · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
GGUF
QW
Qwen3.5 4b Agentic Coder v4
Qwen · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
CodingVision
GU
GUI Owl 1.5 8B Think
Gui · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
ReasoningTool useVision

Other sizes

Keep browsing