Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3 8B
Qwen · Aldaris
Params8.19B
Context40K
Q4_K_M5.0 GB
LA
latxa-7b-v1.2
Latxa · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GE
gemma 4 E4B it heretic
Gemma · Abiray
Params7.5B
Context128K
Q4_K_M5.3 GB
AP
APUS Reflex 9B
Apus · apus-ailab
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
mistralai_mistral-7b-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
TwIL LM3 Pro
Granite · webAI-Official
Params3.66B
Context128K
Q4_K_M2.2 GB
QW
Qwen3 4B
Qwen · lmstudio-community
Params4B
Context32K
Q4_K_M2.5 GB
JE
Jebadiah 9b v2
Jebadiah · frontier-infra
Params9.2B
Context256K
Q4_K_M5.8 GB
OP
openchat-3.6-8b-20240522
Openchat · legraphista
Params8B
Context8K
Smallest2.0 GB
OL
allenai_olmOCR 2 7B 1025
OLMo · lmstudio-community
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Qwen3 Sex
Qwen · ystemsrx
Params4B
Context256K
Smallest4.3 GB
QW
Qwen3.5 4B Claude 4.6 Opus Reasoning Distilled v2
Qwen · Jackrong
Params4.2B
Context256K
Q4_K_M2.7 GB
HU
Hunyuan-MT-7B
Hunyuan · Mungert
Params7.5B
Context256K
Q4_K_M4.7 GB
QW
Qwen Image 2.1 Text Encoder
Qwen · darrellbest
Params8.2B
Context256K
Q4_K_M5.7 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · lmstudio-community
Params7.25B
Context32K
Q4_K_M4.4 GB
FL
mindview-t2i
Flux · mohsenvand
Params4.65B
Context—
Smallest1.2 GB
QW
Qwen3.5 9B Heretic
Qwen · darrellbest
Params9.2B
Context256K
Q4_K_M5.8 GB
QW
Qwen2.5 7B Instruct
Qwen · paultimothymooney
Params7.62B
Context32K
Q4_K_M4.7 GB
IN
InternVL3_5 8B
Internvl · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
GE
gemma 4 E2B it
Gemma · AtomicChat
Params4.65B
Context128K
Q4_K_M3.4 GB
GE
Gemma 2 9b It
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
CA
causallm_7b
TheBloke
Params7.72B
Context8K
Q4_K_M4.8 GB
QW
Qwen3.5 9B Heretic v2
Qwen · filvyb
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen Image 2.1 Turbo Quantized
Qwen · abenzerps
Params7.1B
Context—
Q4_K_M4.1 GB
NE
NeoHorse-1-9B
Neohorse · TokenRhythm
Params8.95B
Context256K
Q4_K_M5.6 GB
DE
deepseek-ai_deepseek-coder-6.7b-base
DeepSeek · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
JA
JackOD 9B Coder
Jackod · ManniX-ITA
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen2.5 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen3.5 9B GLM5.1 Distill v1
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
HO
K2 Horizon 7B Heretic
Horizon · mradermacher
Params9B
Context512K
Q4_K_M5.6 GB