Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
DO
Dolphin v2 8b Abliterated
Dolphin · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
WA
PORNHUBAI
Wan · Boombaaa
Params5.25B
Context—
Q4_K_M3.9 GB
GGUF
QW
asmr qwen3.5 9b zh cn echo gguf v0.1
Qwen · mmis1000
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
LL
Llama 3.2 Taiwan 3B Instruct
Llama · lianghsun
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
ME
q-bert_merged-agi-7b
Merged · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RE
Reggie v3 7b
Reggie · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
ReasoningVision
HU
Huihui Qwopus3.5 9B v3 Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
DR
DrakIdol Roleplayer 1.0
Drakidol · Emerald7664
Params3.88B
Context128K
Q4_K_M2.5 GB
Roleplay
LL
Yarn-Llama-2-7b-128k
Llama · duyntnet
Params6.74B
Context128K
Q4_K_M4.1 GB
GGUF
BU
BuRPInfinWestLakev2 IreneRP Neural 7B slerp
Burpinfinwestlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TE
Tess 4 9B
Tess · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
ReasoningTool useVision
BU
BuddyGlassNeverSleeps methheadmethod v0.2 i1
Buddyglassneversleeps · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
AU
AutoGLM Phone 9B
Autoglm · ggml-org
Params9.4B
Context64K
Q4_K_M6.2 GB
Vision
GE
EZO Common 9B gemma 2 it
Gemma · MCZK
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
NA
Narwhal 7b
Narwhal · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · ggml-org
Params8.03B
Context128K
Smallest6.0 GB
GGUF
OP
open thoughts OpenThinker2 7B
Openthinker · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
neuralmagic Llama 2 7b evolcodealpaca
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
SA
Samantha v5 wizardlm2
Samantha · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 VL 8B Instruct C_Abliterated v3
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
CodingVision
GE
gemma 2 9b it
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
TW
TwinLlama 3.1 8B
Twinllama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
CA
models--mergekit-community--YorkShire11
Carbonbeagle · MaziyarPanahi
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
CA
Calme-7B-Instruct-v0.9
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
CarbonGPT
Chat · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
ThinkLite Critic 7b
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ReasoningVision
NE
intel_neural-chat-7b-v3-3
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · Kondara
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
OR
Orion 9B Agentic CodeCore Merge
Orion · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
LL
llama-3-tulu-2-dpo-8b
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF

Other sizes

Keep browsing