Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
VI
vicuna-7b-v1.5-safeunlearning
Vicuna · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
OC
OceanGPT-7b-v0.1
Oceangpt · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen2-Cantonese-7B-Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Llama3 8b Math Sft
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
EX
Extended-Mega-Mash-262K-8B-L3
Extended · mradermacher
Params8B
Context256K
Q4_K_M4.9 GB
CA
CAT Thinking 8B
Cat · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
LL
LongWriter Llama3.1 8b
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
CA
Calme 7B Instruct v0.3
Calme · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
NO
NovaSpark i1
Novaspark · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen Image 2.1 Turbo Uncensored GGUF BF16
Qwen · KENIC-1
Params8.2B
Context256K
Smallest16.4 GB
ME
Meditron3 8B
Meditron · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
JA
Jais adapted 7B Reflection Tuning Natural Farmer
Jais · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
QW
Qwable 9B Claude Fable 5 Heretic
Qwable · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
MP
Mpt 7b 8k Chat
Mpt · mradermacher
Params6.65B
Context8K
Q4_K_M4.3 GB
KA
KangalKhan Alpha Rubyroid 7B
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
LL
Llama 3.1 EIRAI 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
MI
Mistral-7B-v0.3-Chinese-Chat-uncensored
Mistral · stephenlzc
Params7.25B
Context32K
Q4_K_M4.4 GB
QW
MiroThinker-v1.0-8B
Qwen · Mungert
Params8.2B
Context256K
Q4_K_M5.4 GB
GE
Gemma 4 E4B It Uncensored MAX Opus 4.7
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
SP
Spark X2.6 4B Heretic
Spark · daydreamwarrior
Params4.11B
Context1024K
Smallest3.4 GB
OL
OLMoE-1B-7B-0125-Instruct-Distill-ot114k-batch32
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
ZE
huggingfaceh4_zephyr-7b-alpha
Zephyr · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
OM
OmniCoder 2 9B
Omnicoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
Gukbap Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 Coder 7B Instruct GGUF
Qwen · simon5133
Params7.62B
Context128K
Q4_K_M9.4 GB
HE
calderaai_hexoteric-7b
Hexoteric · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
PII Model Phi3 Mini
Phi · mradermacher
Params3.82B
Context4K
Q4_K_M2.4 GB
ST
starcoderbase 7b
StarCoder · QuantFactory
Params7.5B
Context8K
Q4_K_M4.9 GB
OC
octo net
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB