Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
DO
Dolphin v2 8b Abliterated
Dolphin · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
WA
PORNHUBAI
Wan · Boombaaa
Params5.25B
Context—
Q4_K_M3.9 GB
QW
asmr qwen3.5 9b zh cn echo gguf v0.1
Qwen · mmis1000
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama 3.2 Taiwan 3B Instruct
Llama · lianghsun
Params3.61B
Context128K
Q4_K_M2.2 GB
ME
q-bert_merged-agi-7b
Merged · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
RE
Reggie v3 7b
Reggie · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
HU
Huihui Qwopus3.5 9B v3 Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
DR
DrakIdol Roleplayer 1.0
Drakidol · Emerald7664
Params3.88B
Context128K
Q4_K_M2.5 GB
LL
Yarn-Llama-2-7b-128k
Llama · duyntnet
Params6.74B
Context128K
Q4_K_M4.1 GB
BU
BuRPInfinWestLakev2 IreneRP Neural 7B slerp
Burpinfinwestlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
TE
Tess 4 9B
Tess · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
BU
BuddyGlassNeverSleeps methheadmethod v0.2 i1
Buddyglassneversleeps · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
AU
AutoGLM Phone 9B
Autoglm · ggml-org
Params9.4B
Context64K
Q4_K_M6.2 GB
GE
EZO Common 9B gemma 2 it
Gemma · MCZK
Params9.24B
Context8K
Q4_K_M5.8 GB
NA
Narwhal 7b
Narwhal · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
Meta Llama 3.1 8B Instruct
Llama · ggml-org
Params8.03B
Context128K
Smallest6.0 GB
OP
open thoughts OpenThinker2 7B
Openthinker · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
neuralmagic Llama 2 7b evolcodealpaca
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
SA
Samantha v5 wizardlm2
Samantha · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 VL 8B Instruct C_Abliterated v3
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
GE
gemma 2 9b it
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
TW
TwinLlama 3.1 8B
Twinllama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
CA
models--mergekit-community--YorkShire11
Carbonbeagle · MaziyarPanahi
Params7.24B
Context4K
Q4_K_M4.4 GB
CA
Calme-7B-Instruct-v0.9
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CH
CarbonGPT
Chat · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
ThinkLite Critic 7b
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
NE
intel_neural-chat-7b-v3-3
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · Kondara
Params7.62B
Context128K
Q4_K_M4.7 GB
OR
Orion 9B Agentic CodeCore Merge
Orion · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
llama-3-tulu-2-dpo-8b
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB