Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
CY
BaronLLM Offensive Security abliterated
Cybersecurity · huihui-ai
Params8B
Context—
Smallest6.6 GB
GGUF
QW
Qwen3 Esper3 Reasoning Instruct 6B Brainstorm20x Enhanced E32
Qwen · mradermacher
Params5.94B
Context40K
Q4_K_M3.6 GB
CodingReasoning
OP
openPangu Embedded 7B V1.1
Openpangu · noctrex
Params8.03B
Context32K
Q4_K_M4.9 GB
Embedding
LL
Llama 3.1 Nemotron Nano 4B v1.1
Llama · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
GGUF
LL
Configurable Llama 3.1 8B Instruct i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
ZW
ZwZ2 4B
Zwz · bartowski
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
GL
Glm 4 9b Chat 1m
GLM · bartowski
Params9.48B
Context1024K
Q4_K_M6.3 GB
GGUF
MI
speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Bio Medical Llama 3 8B i1
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TR
Trurl 2 7B
Trurl · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NE
models--automerger--NeuralsynthesisT3qm7-7B
Neuralsynthesis · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma The Writer 9B HERETIC Uncensored Abliterated
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
Roleplay
FU
functionary small v3.2
Functionary · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
AL
allknowingroger CalmExperiment 7B slerp
Allknowingroger · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama2 7b chat codeCherryPop qLoRA
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
RU
RusMind Vspyshka 4.7
Rusmind · DICPlay
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
TR
Transliteration 2.0 4B Safetensors
Transliteration · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
MI
mistral Summarizer 7b instruct v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
Intern-S1-Mini
InternLM · internlm
Params8.2B
Context40K
Smallest8.7 GB
ReasoningTool useVision
CO
ComfyUI
Comfyui · f5aiteam
Params7.62B
Context125K
Q4_K_M4.7 GB
GGUF
AU
AuroraCap 7B IMG xtuner i1
Auroracap · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
adamo1139_mistral-7b-aezakmi-v1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
Configurable-Yi-1.5-9B-Chat
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
VI
vicuna 7B v1.5
Vicuna · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NE
Dory1.0 4B BF16 Llamacpp
Nemotron · cloudnathan5
Params3.98B
Context256K
Q4_K_M2.4 GB
GGUF
LL
hon9kon9ize Cantonese Llama 2 7B preview20240625
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DA
Dans PersonalityEngine v1.0.0 8b
Dans · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
Coding
NE
NeuralArjuna 7B DT
Neuralarjuna · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen3 VL 8B Heretic Stable
Qwen · prithivMLmods
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision

Other sizes

Keep browsing