Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Llama 3.2 1B Instruct Gsm8K
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GR
granite vision crispembed
Granite · cstr
Params—
Context—
Smallest0.0 GB
MI
cloudyu Mixtral 7Bx2 MoE 13B
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
MU
Muse Glimmer 30B Surgical 6p34 Q4 K recovered
Muse · Joakimpalm-Zen
Params27.9B
Context128K
Smallest15.7 GB
QW
qwen2.5 vl 3b crispembed
Qwen · cstr
Params3.75B
Context—
Smallest2.7 GB
LL
Llama 3.1 8B InitializedEmbeddings_with_Hermes 3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
SA
SambaLingo Arabic Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
MI
arnavgrg mistral 7b nf4 fp16 upscaled
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SA
saiga2 13b
Saiga · IlyaGusev
Params13B
Context2K
Smallest5.4 GB
AR
ArianAskari SOLID SFT DPO MixQV3 SOLIDChosen SFTRejected Zephyr 7b beta
Arianaskari · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
KSU HW SEC Deepseek SFT RL Text DFG
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
FA
FaizanPervaz stablecode alpha 3b finetuned
Faizanpervaz · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
OP
OpenHermes-2.5-Nebula-v2-7B
Openhermes · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PA
Pastiche Crown Clown 7b Dare
Pastiche · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
codellama CodeLlama 7b Python hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Phi 3 Mini 128k Instruct LLaMAfied
Llama · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
LL
abhishek2475 Llama 3.2 3B Instruct finetuned
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
ME
mergekit community Mathing 1.5B
Mergekit · RichardErkhov
Params1.59B
Context32K
Q4_K_M1.0 GB
MI
Mistral_Pro_8B_v0.1
Mistral · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
QW
abdulmannan 01 qwen 2.5 3b finetuned for sql generation
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
FU
FuseChat 7B v2.0
Fusechat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
LDCC Instruct Llama 2 Ko 13B v4.1.8
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
LL
GNER LLaMA 7B
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
FINGU AI Qwen2.5 14B Instruct Fine Tuned v3
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
MI
mlfoundations dev hp ablations mistral adambeta1 0.85 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
MI
mlfoundations dev hp ablations grid mistral bsz512 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
QW
Qwen2.5 1b
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB