Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
mimo tokenizer
Mimo · cstr
Params649M
Context—
Smallest0.4 GB
GGUF
LL
Llama 3 Tulu 2 70b
Llama · RichardErkhov
Params70.6B
Context2K
Q4_K_M42.5 GB
GGUF
LL
Llama 2 7b Pruned50 Retrained
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
KU
Kukedlc SuperMente 7B v4
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
mango qwen35 122b q4
Qwen · isilicon
Params122B
Context256K
Smallest74.2 GB
ReasoningTool use
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
kykim0 Llama 2 7b ultrachat syn
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b1
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
SA
SanGuoYanYi 6b
Sanguoyanyi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
GL
GLM 4.7 Flash abliterated 12GB
GLM · cognitivers
Params29.9B
Context198K
Smallest0.1 GB
GGUF
LL
KISTI KONI KONI Llama3 8B Merged 20240724
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
read1337 Llama 3.1 8B bnb 4bit marketing
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
CH
chatterbox nano
Chatterbox · cstr
Params180M
Context—
Smallest0.3 GB
GGUF
FL
FLUX.1 schnell Quantized
Flux · aifoundry-org
Params11.9B
Context—
Smallest3.9 GB
GGUF
CO
Yi Coder 9B
Coder · legraphista
Params8.83B
Context128K
Smallest2.0 GB
Coding
GR
Granite 7b Instruct
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
PatronusAI Llama 3 Patronus Lynx 70B Instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
DE
Huihui DeepSeek V4 Flash Vision Exp abliterated
DeepSeek · taurusduan
Params284B
Context—
Smallest135.3 GB
GGUF
RE
reazonspeech nemo v2
Reazonspeech · cstr
Params619M
Context—
Smallest0.4 GB
GGUF
MI
m8than mistral 7b instruct 0.2
Mistral · RichardErkhov
Params7.24B
Context—
Q4_K_M1.1 GB
GGUF
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DY
DynaGuard AIO
Dynaguard · prithivMLmods
Params1.72B
Context40K
Q4_K_M1.1 GB
ReasoningTool use
MI
MiniCPM5 1B
MiniCPM · eaddario
Params1.08B
Context128K
Smallest0.0 GB
GGUF
LL
Llama 3 8B ArliAI Formax v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NO
Noromaid-7B-0.4-DPO
Noromaid · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RO
RoLlama2-7b-Chat
Rollama · legraphista
Params6.74B
Context4K
Smallest1.5 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Mrt45
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
MI
Voxtral-Mini-4B-Realtime-2602
Mistral · mistral-experimental
Params4B
Context—
Smallest3.1 GB
GGUF
LL
Llama 3 8B Instruct 262k
Llama · Monor
Params8.03B
Context256K
Q4_K_M4.9 GB
GGUF
DE
DeepCoder 14B Preview
Deepcoder · Joumdane
Params14.8B
Context128K
Q4_K_M9.0 GB
CodingReasoningTool use

Other quants

Keep browsing