Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
MiMo V2.6 Distill Qwen 9B Ablit
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool use
TU
Tulpar 7B v0
Tulpar · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3-8B-abliterated
Qwen · Mungert
Params8.2B
Context40K
Q4_K_M5.2 GB
GGUF
QW
Qwen3 6B Hivemind Instruct Heretic Abliterated Uncensored NEO Imatrix
Qwen · DavidAU
Params5.94B
Context256K
Q4_K_M3.6 GB
Roleplay
IN
models--automerger--Inex12Yam-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
Fara 7B
Fara · gguf-org
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
AP
Apocrypha-7b
Apocrypha · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama3-8B-Instruct-Replete-Adapted
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CodingTool use
DE
deepseek llm 7B base
DeepSeek · TheBloke
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
NE
Huihui NVIDIA Nemotron Nano 9B v2 Abliterated
Nemotron · mradermacher
Params8.89B
Context1024K
Q4_K_M6.5 GB
GGUF
QW
Qwen3 4B Qwen3.6 plus Reasoning Distilled
Qwen · khazarai
Params4B
Context256K
Smallest2.6 GB
Reasoning
LL
LLaMAX2-7B-MetaMath
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 4B Python Coder
Qwen · Abiray
Params4.2B
Context256K
Q4_K_M2.7 GB
Coding
LL
Llama 3.3 8B Instruct 128K Jbliterated
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LE
ignos_leoscorpius-greennode-platypus-7b-v1
Leoscorpius · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
X Coder RL Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
Coding
OP
q-bert_optimus-7b
Optimus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
granite 4.2 3b heretic
Granite · saidutta69
Params3.66B
Context128K
Q4_K_M2.2 GB
ReasoningRoleplayTool use
MI
Mixtral AI CyberRolePlay Orca
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
MI
nruaif_kimiko-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
uukuguy_speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
HE
L3.1 RP Hero InBetween 8B
Hero · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
IC
IceEspressoRPv2 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
teknium_mistral-trismegistus-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-3-Groq-8B-Tool-Use
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
Tool use
MI
janhq_mistral-7b-instruct-v0.2-dare
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SW
Swallow 7B
Swallow · TheBloke
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama 3.1 8B Ultra Instruct
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
JE
Jebadiah 4b v2
Jebadiah · frontier-infra
Params4.33B
Context256K
Q4_K_M2.8 GB
GGUF
QW
X Coder SFT Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding

Other sizes

Keep browsing