Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
SA
Saul 7B Base
Saul · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AH
Ahma 3B Instruct
Ahma · RichardErkhov
Params3.63B
Context2K
Q4_K_M2.8 GB
GGUF
SA
SakanaAI EvoLLM JP A v1 7B
Sakanaai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HO
hooking dev sexyGPT Uncensored
Hooking · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · SanctumAI
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
CA
cais Zephyr RMU
Cais · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
jointpreferences mistral 7b sft helpful
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 4B
Qwen · eaddario
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
QW
VanillaKD Pretrain Qwen 200M
Qwen · RichardErkhov
Params203M
Context32K
Q4_K_M0.2 GB
GGUF
IN
MD Judge V0_2 Internlm2_7B
InternLM · RichardErkhov
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
CL
clipclap
Clipclap · antflydb
Params69M
Context—
Smallest0.0 GB
GGUF
CH
chujiezheng tulu 2 dpo 13b ExPO
Chujiezheng · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
GGUF
GE
Gemma4 26B A4B Instruct QAT
Gemma · Atomic-Germ
Params25.2B
Context256K
Smallest0.1 GB
GGUF
JA
Jan v1 AIO
Jan · prithivMLmods
Params4.02B
Context40K
Q4_K_M1.1 GB
ReasoningTool use
SA
Salamandra 2b
Salamandra · RichardErkhov
Params2.25B
Context8K
Q4_K_M1.5 GB
GGUF
OC
PP-OCRv6_medium_det
Ocrv · cstr
Params22M
Context—
Smallest0.0 GB
GGUF
AS
x-asr-zh-en
Asr · cstr
Params153M
Context—
Smallest0.1 GB
GGUF
GE
Gargaz gemma 2b romanian better
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
NE
NexaAIDev Octopus v2
Nexaaidev · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen3.5 4B
Qwen · ZuzeTt
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
LL
Cendol Llama2 13b Merged
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
CO
Yi Coder 1.5B Chat
Coder · legraphista
Params1.48B
Context128K
Smallest0.5 GB
Coding
CY
z image cyberrealistic v4
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
CR
crackthejob codingLLM
Crackthejob · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama-1.1B-step-50K-105b
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Na0s Llama 3.1 8B Pruned 4 Layers LoRA PEFT 3.0
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
GGUF
QW
Qwen3.8-27B
Qwen · Sh200126
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
LL
winglian llama 3 8b 256k PoSE
Llama · RichardErkhov
Params8.03B
Context512K
Q4_K_M4.9 GB
GGUF
WA
wav2vec2 large xlsr 53 french
Wav · cstr
Params350M
Context—
Smallest0.0 GB
GGUF
OR
orpheus 3b base
Orpheus · cstr
Params3.3B
Context—
Smallest2.6 GB
GGUF

Other quants

Keep browsing