Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LA
Lawma 8b
Lawma · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
CH
chinese alpaca 2 1.3b rlhf
Chinese · hfl
Params1.26B
Context4K
Smallest0.6 GB
GGUF
MA
magnum v2 12b
Magnum · anthracite-org
Params12.2B
Context1000K
Smallest5.7 GB
GGUF
NE
Ea_Final_Nvidia_Nemotron_3_Super_120B_A12B_Bf16_030326_Vv0.1
Nemotron · ggml-org
Params121B
Context1024K
Smallest69.9 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
SE
SeaLLM 7B Chat
Seallm · parinzee
Params6.87B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
TI
TinySlime-1.1B-v1.0
Tinyslime · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
GP
GPT OSS Cybersecurity Merged 20B Heretic Ara
gpt-oss · piotreknow02
Params20.9B
Context128K
Q4_K_M15.8 GB
Tool use
WA
NSFW Wan 14b video
Wan · Slinkies86
Params14.3B
Context—
Smallest9.7 GB
GGUF
LL
nirosunid Llama 3.1 8B bnb 4bit python v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
FL
Flux2 Klein 9B True V2
Flux · Leon1000
Params9.08B
Context—
Smallest5.7 GB
GGUF
WA
wav2vec2 large xlsr 53 german
Wav · cstr
Params315M
Context—
Smallest0.2 GB
GGUF
LL
Llama-3-Instruct-8B-SimPO
Llama · legraphista
Params8.03B
Context8K
Smallest2.0 GB
GGUF
LL
llm-compiler-7b
Llm · legraphista
Params6.74B
Context16K
Smallest1.5 GB
GGUF
QW
Qwen3.6 27B Uncensored Genesis Final
Qwen · LuffyTheFox
Params26.9B
Context256K
Smallest17.5 GB
ReasoningTool useVision
QW
Qwen Image 2.1
Qwen · hhfw
Params8.19B
Context256K
Smallest3.3 GB
CodingTool useVision
ML
mlfoundations dev stackoverflow 10000tasks 0p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
ST
statking zephyr 7b sft full orpo
Statking · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
1 800 SHARED TASKS PHI3 Medium NLI 16bit
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
GGUF
BG
bge m3
BGE · cstr
Params567M
Context8K
Smallest0.4 GB
Embedding
MI
Mistral 7B Merge 14 v0.1
Mistral · LiteLLMs
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SU
Sungur 14B GGUF
Sungur · suayptalha
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningTool use
DM
dmis lab meerkat 7b v1.0
Dmis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated v3
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
GE
gemma-4-E2B-it
Gemma · cstr
Params4.95B
Context—
Smallest1.6 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · nurdich
Params1.86B
Context256K
Smallest0.9 GB
GGUF
LI
Ming Image 0.1 Design
Ling · voltaire321
Params6.15B
Context—
Smallest6.5 GB
GGUF
FL
Flux2 Klein 9B True V2
Flux · BIGJUTT
Params9.08B
Context—
Smallest5.7 GB
GGUF

Other quants

Keep browsing