Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
OL
OLMoE 1B 7B 0125 Instruct
OLMo · allenai
Params6.9B
Context4K
Q4_K_M4.2 GB
GGUF
RY
RynnUnify Agentic Reasoner
Rynnunify · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Tool use
MI
jphme_em_german_leo_mistral
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
ComfyUI Qwen3 VL
Qwen · Abiray
Params8.77B
Context256K
Q4_K_M2.5 GB
Tool useVision
OP
openchat_openchat-3.5-0106
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
chronos mistral 7b i1
Mistral · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 8B ERP v0.1
Qwen · Aratako
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
MI
mimo asr
Mimo · cstr
Params8.02B
Context—
Smallest0.0 GB
GGUF
KA
KAI 7B Instruct v0.1
Kai · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
VE
Veritas 8B Fact Checker Non Thinking 1.0
Veritas · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Reasoning
LL
model
Llama · ChatGpt1
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 7B Instruct
Qwen · CISCai
Params7.62B
Context32K
Smallest1.9 GB
Coding
HU
HumanlikeRP
Humanlikerp · mradermacher
Params8.83B
Context12K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.5 4B EmperoAI Qwen3.8 Distill Heretic Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
MI
paixai_astrid-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
LlamaScientific 8B
Llama · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
CodingVision
QW
Co Rewarding I Qwen3 8B Base MATH
Qwen · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
GGUF
QW
Qwen Image 2.1 Uncensored
Qwen · basileleuwa
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
SO
Socio Foundation 8B
Socio · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
RO
L3.2 Rogue Creative Instruct Uncensored 7B
Rogue · DavidAU
Params7.53B
Context128K
Q4_K_M4.6 GB
Roleplay
DA
L3.1 Dark Planet SpinFire Uncensored 8B
Dark · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
AI
airoboros l2 7b gpt4 1.4.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CL
CLM v0.1 8B
Clm · czl
Params8.19B
Context40K
Q4_K_M5.0 GB
Embedding
QW
_merged_Crow 4B Opus 4.6 Distill Heretic_Qwen3.5
Qwen · Crownelius
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingReasoningVision
IN
InnerIAI chat 7b grok
Inneriai · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
GE
Tiger-Gemma-9B-v1
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
FL
Flux2 Klein 4b Uncensored Text Encoder
Flux · nerualdreming
Params4B
Context40K
Q4_K_M2.5 GB
Coding
LL
NousResearch DeepHermes 3 Llama 3 8B Preview
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplayTool use
LL
saiga_llama3_8b
Llama · IlyaGusev
Params8B
Context8K
Smallest3.2 GB
GGUF

Other sizes

Keep browsing