Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
LL
CodeLlama 7b Instruct hf
Llama · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
DE
WhiteRabbitNeo V3 7B i1
Deephat · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
NE
models--automerger--NeuralsynthesisMultiverseex26-7B
Neuralsynthesis · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
TH
The Trinity Coder 7B
The · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MO
models--automerger--T3qT3qm7xp-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
nickynicky_mistral-7b-openorca-oasst_top1_2023-08-25-v2
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
LevelField
Levelfield · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Turkish Llama 8b Instruct v0.1
Llama · ytu-ce-cosmos
Params8B
Context8K
Q4_K_M4.9 GB
LL
Llama 3 8B Instruct Gradient 1048k GGUF smashed
Llama · PrunaAI
Params8B
Context1024K
Q4_K_M4.9 GB
PH
Iceball Phi 3.5 mini instruct shareGPT v1.0 i1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
QW
Git Commit Message Splitter Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
QW
Qwen Image 2.1 Uncensored
Qwen · 0Endzz
Params7.12B
Context—
Q4_K_M4.6 GB
QW
Qwen3 DND Jan v1 256k Ctx Brainstorm40x 8B
Qwen · mradermacher
Params7.66B
Context256K
Q4_K_M4.7 GB
IC
IceEspressoRPv1 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Replete LLM V2.5 Qwen 7b
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Fallen Gemma3 4B v1
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
AG
Agents A1 4B
Agents · bartowski
Params4.21B
Context256K
Q4_K_M2.9 GB
JA
Jais-3-7B
Jais · IFM
Params9B
Context512K
Q4_K_M5.6 GB
ST
Step3 VL 10B
Step · seanbailey518
Params8.19B
Context64K
Q4_K_M5.0 GB
QW
Qwen3.5-9B
Qwen · EcoAI-Lab
Params9.2B
Context256K
Smallest3.9 GB
WE
WestLakev2 IreneRP Neural 7B slerp
Westlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
VI
Video ORA 9B
Video · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
QW
typecastlm qwen3.5 3.8b
Qwen · mihailgribov
Params3.76B
Context256K
Smallest4.0 GB
AP
Apertus SEA LION v4 8B IT
Apertus · aisingapore
Params8.05B
Context64K
Q4_K_M5.1 GB
IR
IreneRP Neural 7B slerp
Irenerp · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
YI
Yi-1.5-9B
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
KA
KangalKhan RawRuby 7B
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
madlad400 7b mt
Madlad · google
Params8.3B
Context—
Smallest4.7 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Q4_K_M4.7 GB