Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
GL
Poe 8B GLM5 Opus4.6 Sonnet4.5 Kimi Grok Gemini 3 Pro Preview HERETIC
GLM · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
LA
law chat
Law · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
JA
JapanRailwaysimLLM
Japanrailwaysimllm · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Amory0201
Params7.5B
Context128K
Q4_K_M5.3 GB
HU
Huihui4 8B A4B v2
Huihui · mradermacher
Params8.1B
Context256K
Q4_K_M5.4 GB
LL
Llama 3.3 8B Instruct
Llama · Green-Eye
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3-Reranker-4B
Qwen · gorse-io
Params4.02B
Context40K
Q4_K_M2.5 GB
LL
llama-3-Nephilim-v2-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
PH
Phi 3 mini 4k instruct imatrix
Phi · duyntnet
Params3.82B
Context4K
Q4_K_M2.4 GB
HA
Hathor v4.6 i1
Hathor · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
AU
AutoCoder S 6.7B
Autocoder · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Llama-3-Oasis-v1-OAS-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CO
Cosmos Reason1 7B Heretic
Cosmos · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
LL
Meta Llama 3.1 8B Instruct
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
LL
Llama 3 Youko 8b Instruct
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 Coder 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma4 E4B Mahou Nsfw
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
PY
Pythia 6.9b Deduped
Pythia · mradermacher
Params6.86B
Context2K
Q4_K_M4.4 GB
QW
Qwen3.5 9B Cyberpunk2077 Xiaotian All4 5pass
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
OP
Openhands Lm 7b v0.1
Openhands · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
DE
Deepseek Llm 7b Chat Abliterated
DeepSeek · mradermacher
Params6.91B
Context4K
Q4_K_M4.2 GB
IG
Ignis 7B DPO Laser
Ignis · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
NE
Daredevil-8B-abliterated-dpomix
Neuraldaredevil · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
AM
amalia-desenrasca-9b
Amalia · mradermacher
Params9.15B
Context32K
Q4_K_M5.6 GB
LL
Llama-2-7b-pruned70-retrained-gsm8k
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ME
MedMO 8B Next
Medmo · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · lmstudio-community
Params7.62B
Context4K
Q4_K_M4.7 GB