Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
VI
vicuna-7b-v1.5-safeunlearning
Vicuna · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OC
OceanGPT-7b-v0.1
Oceangpt · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen2-Cantonese-7B-Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Llama3 8b Math Sft
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
EX
Extended-Mega-Mash-262K-8B-L3
Extended · mradermacher
Params8B
Context256K
Q4_K_M4.9 GB
GGUF
CA
CAT Thinking 8B
Cat · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
LL
LongWriter Llama3.1 8b
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
CA
Calme 7B Instruct v0.3
Calme · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NO
NovaSpark i1
Novaspark · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
QW
Qwen Image 2.1 Turbo Uncensored GGUF BF16
Qwen · KENIC-1
Params8.2B
Context256K
Smallest16.4 GB
CodingVision
ME
Meditron3 8B
Meditron · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
JA
Jais adapted 7B Reflection Tuning Natural Farmer
Jais · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
GGUF
QW
Qwable 9B Claude Fable 5 Heretic
Qwable · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MP
Mpt 7b 8k Chat
Mpt · mradermacher
Params6.65B
Context8K
Q4_K_M4.3 GB
Coding
KA
KangalKhan Alpha Rubyroid 7B
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
LL
Llama 3.1 EIRAI 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Mistral-7B-v0.3-Chinese-Chat-uncensored
Mistral · stephenlzc
Params7.25B
Context32K
Q4_K_M4.4 GB
Coding
QW
MiroThinker-v1.0-8B
Qwen · Mungert
Params8.2B
Context256K
Q4_K_M5.4 GB
GGUF
GE
Gemma 4 E4B It Uncensored MAX Opus 4.7
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
SP
Spark X2.6 4B Heretic
Spark · daydreamwarrior
Params4.11B
Context1024K
Smallest3.4 GB
CodingReasoningTool use
OL
OLMoE-1B-7B-0125-Instruct-Distill-ot114k-batch32
OLMo · mradermacher
Params6.9B
Context4K
Q4_K_M4.2 GB
Reasoning
ZE
huggingfaceh4_zephyr-7b-alpha
Zephyr · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OM
OmniCoder 2 9B
Omnicoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingVision
MI
Gukbap Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct GGUF
Qwen · simon5133
Params7.62B
Context128K
Q4_K_M9.4 GB
CodingTool use
HE
calderaai_hexoteric-7b
Hexoteric · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
PII Model Phi3 Mini
Phi · mradermacher
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
ST
starcoderbase 7b
StarCoder · QuantFactory
Params7.5B
Context8K
Q4_K_M4.9 GB
Coding
OC
octo net
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF

Other sizes

Keep browsing