Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
MiMo V2.6 Distill Qwen 9B Ablit
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
TU
Tulpar 7B v0
Tulpar · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen3-8B-abliterated
Qwen · Mungert
Params8.2B
Context40K
Q4_K_M5.2 GB
QW
Qwen3 6B Hivemind Instruct Heretic Abliterated Uncensored NEO Imatrix
Qwen · DavidAU
Params5.94B
Context256K
Q4_K_M3.6 GB
IN
models--automerger--Inex12Yam-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
FA
Fara 7B
Fara · gguf-org
Params7.62B
Context125K
Q4_K_M4.7 GB
AP
Apocrypha-7b
Apocrypha · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama3-8B-Instruct-Replete-Adapted
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
DE
deepseek llm 7B base
DeepSeek · TheBloke
Params6.91B
Context4K
Q4_K_M4.2 GB
NE
Huihui NVIDIA Nemotron Nano 9B v2 Abliterated
Nemotron · mradermacher
Params8.89B
Context1024K
Q4_K_M6.5 GB
QW
Qwen3 4B Qwen3.6 plus Reasoning Distilled
Qwen · khazarai
Params4B
Context256K
Smallest2.6 GB
LL
LLaMAX2-7B-MetaMath
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen3.5 4B Python Coder
Qwen · Abiray
Params4.2B
Context256K
Q4_K_M2.7 GB
LL
Llama 3.3 8B Instruct 128K Jbliterated
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
LE
ignos_leoscorpius-greennode-platypus-7b-v1
Leoscorpius · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
X Coder RL Qwen3 8B
Qwen · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
OP
q-bert_optimus-7b
Optimus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
granite 4.2 3b heretic
Granite · saidutta69
Params3.66B
Context128K
Q4_K_M2.2 GB
MI
Mixtral AI CyberRolePlay Orca
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
nruaif_kimiko-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
uukuguy_speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
HE
L3.1 RP Hero InBetween 8B
Hero · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
IC
IceEspressoRPv2 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
teknium_mistral-trismegistus-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-3-Groq-8B-Tool-Use
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
MI
janhq_mistral-7b-instruct-v0.2-dare
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
SW
Swallow 7B
Swallow · TheBloke
Params6.83B
Context4K
Q4_K_M4.1 GB
LL
Llama 3.1 8B Ultra Instruct
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
JE
Jebadiah 4b v2
Jebadiah · frontier-infra
Params4.33B
Context256K
Q4_K_M2.8 GB
QW
X Coder SFT Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB