Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
DE
Srijith rkr deepseek base 1e 3 NO cot all samples 3 epoch
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GR
Granite 3.0 8b Instruct
Granite · RichardErkhov
Params8.17B
Context4K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama 1.1B Intermediate Step 1431k 3T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
GR
GritLM 8x7B
Gritlm · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
SA
SambaLingo Arabic Base 70B
Sambalingo · RichardErkhov
Params69.4B
Context4K
Q4_K_M41.7 GB
GGUF
MI
Mistral 7B Instruct v0.2 Function Calling
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
QwQ 32B
Qwen · dahara1
Params32.8B
Context128K
Q4_K_M19.9 GB
ReasoningTool use
SM
SmolLM3 3B
SmolLM · Bastion-AI
Params3.08B
Context64K
Q4_K_M1.9 GB
ReasoningTool use
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
QW
elinas Qwen2 11.3B
Qwen · RichardErkhov
Params11.3B
Context128K
Q4_K_M6.9 GB
GGUF
QW
cofeg Finetuned Xunzi Qwen2 1.5B for ancient text generation
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
MA
magnum v2 32b
Magnum · anthracite-org
Params32.5B
Context32K
Smallest14.7 GB
GGUF
EX
Excalibur 7b
Excalibur · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 7b
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
DE
AymanTarig deepseek coder 1.3b FC v2
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
DO
DongfuJiang prm version2 subsample hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
kimleang123 qwen 7b v2 khmer qa 1000 v10 merge model attn flash 2
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
KO
JY623 KoSOLAR 10.7B merge v3.0
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
ruslandev llama 3 8b gpt 4o ru1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
SFR Iterative DPO LLaMA 3 8B R
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GL
Glm 4 9b Chat
GLM · RichardErkhov
Params9.4B
Context128K
Q4_K_M6.3 GB
Tool use
LL
yujiepan llama 3 tiny random
Llama · RichardErkhov
Params14M
Context—
Q4_K_M0.0 GB
GGUF
SM
akhilfau fine tuned smolLM2 physics 10
SmolLM · RichardErkhov
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
IN
internlm2-limarp-chat-20b
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
GE
Gemma 2 2b Jpn It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
BL
blue pencil flux1
Blue · bluepen5805
Params11.9B
Context—
Q4_K_M6.9 GB
GGUF
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
GP
gpt2-small-italian-embeddings
Gpt · RichardErkhov
Params132M
Context1K
Q4_K_M0.1 GB
Embedding
CP
cpt_st-vicuna-v1.3-1.5b-ppl
Cpt · RichardErkhov
Params1.48B
Context2K
Q4_K_M0.9 GB
GGUF

Other quants

Keep browsing