Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Nsfwvision_V5
Qwen · GitMylo
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
LL
Llama3.1 8B PlumCode
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Coding
HE
L3.1 RP Hero BigTalker 8B
Hero · DavidAU
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
PH
Phi 3 mini 128k instruct
Phi · mmnga
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
Roleplay-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
LF
LFM2.5-8B-A1B-Uncensored-Gaston
Lfm · Justbackup
Params8.47B
Context125K
Q4_K_M5.2 GB
Reasoning
NA
Nanbeige4.1 3B
Nanbeige · mradermacher
Params3.93B
Context256K
Q4_K_M2.4 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplay
PH
Phi 3.5 Mini ChatML
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
PH
Phi 3.5 Mini instruct_Uncensored
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
QW
Qwen3 4B Uncensored
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
BA
BananaMind Translate V3.5 Pro
Bananamind · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
GGUF
OR
Orpheus 3b FT Q8 0
Orpheus · lex-au
Params3.78B
Context128K
Smallest4.0 GB
Tool use
OP
weyaxi_openhermes-2.5-neural-chat-v3-3-slerp
Openhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MA
Magnum v4 9b
Magnum · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
QW
Qwen3 8B Abliterated
Qwen · DuoNeural
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
NO
Nous Capybara 7B
Nous · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CO
ContextualKunoichi KTO 7B
Contextualkunoichi · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma-4-E2B-It
Gemma · UnitGrief
Params4.65B
Context128K
Q4_K_M3.1 GB
Vision
OC
OculusMind ToolCall 8B v2
Oculusmind · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
Tool use
TY
scb10x_typhoon-7b
Typhoon · TheBloke
Params7.27B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Logical-elyza-llama2-7b-fast-instruct
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
CodingRoleplay
QW
VideoGuard Qwen3.5 9B Safety RL Uncensored
Qwen · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.1 8B Instruct Heretic V1 1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
GEITje 7b Uncensored
Geitje · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Huihui Qwen3 8B Abliterated v2
Qwen · dcv21
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
NE
NVIDIA Nemotron Nano 9B v2 Japanese
Nemotron · mradermacher
Params8.89B
Context1024K
Q4_K_M6.5 GB
GGUF
CH
Chocolatine 3B Instruct DPO v1.0 i1
Chocolatine · mradermacher
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF

Other sizes

Keep browsing