Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
NO
nomic-embed-code
Nomic · nomic-ai
Params7.1B
Context32K
Q4_K_M4.4 GB
CodingEmbedding
US
UserLM 8b
Userlm · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Docto Decision Gemma 4 E4B Fr v0.1
Gemma · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
JA
Jais-3-3.7B
Jais · IFM
Params5.06B
Context512K
Q4_K_M3.2 GB
ReasoningTool use
QW
Qwen2.5 Coder 7B
Qwen · ggml-org
Params7.62B
Context32K
Smallest8.1 GB
Coding
QW
Qwen3.8 Flash Next MTP
Qwen · quimmedes
Params3.88B
Context256K
Smallest1.2 GB
GGUF
TH
TheSpice 7b FT ExperimentalOrca
Thespice · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
OP
OpenCodeInterpreter-DS-6.7B
Opencodeinterpreter · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
DA
pocketdoc_dans-adventurouswinds-mk2-7b
Dans · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
OpenChat 3.5 7B Mixtral v2.0 i1
Mistral · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
nurtureai_openhermes-2.5-mistral-7b-16k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
DE
Decka 4B
Decka · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
QW
GPT 5 Distill Qwen3 4B Instruct
Qwen · Jackrong
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
LL
Llama 2 7b WikiChat
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
NA
NaNovel 9B
Nanovel · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
US
UserRepGen 4B
Userrepgen · bartowski
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
OM
OmniCoder 9B
Omnicoder · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Coding
HU
HuatuoGPT 3 9B
Huatuogpt · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
LL
Trendyol LLM 8b Chat v2.0
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwopus3 5 9B Coder Flash
Qwopus · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
QW
Qwen3 VL 8B Thinking
Qwen · bartowski
Params8.19B
Context256K
Q4_K_M5.0 GB
ReasoningVision
LL
Llama 3.1 Nemotron Nano 8B v1
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
RU
RusMind.Моментум 3.5.1
Rusmind · DICPlay
Params4.65B
Context128K
Q4_K_M3.4 GB
ReasoningTool use
MI
ehartford_samantha-1.2-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Herplete LLM Llama 3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
SE
sciphi_sensei-7b-v1
Sensei · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
teknium_collectivecognition-v1.1-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OR
Cyber Ornith 1.5 9B OBLITERATED
Ornith · DuoNeural
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
AI
Aitana 7B S Instruct
Aitana · mradermacher
Params7.77B
Context8K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing