Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
MiniMoog Mergerix 7b v0.3
Minimoog · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
peft internal testing tiny dummy qwen2
Qwen · RichardErkhov
Params11M
Context—
Q4_K_M0.0 GB
LL
Dumi Llama3.2 3b Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
FA
ssmits Falcon2 5.5B Dutch
Falcon · RichardErkhov
Params5.47B
Context2K
Q4_K_M3.4 GB
LL
lianghsun Llama 3.2 Taiwan 3B
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
MU
mudler Asinello Minerva 3B v0.1
Mudler · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
NU
Nutanix checkpoint 1800 merged
Nutanix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
tinyllama-1.1B-intermediate-step-715k-1.5T-dpo-lora-merged
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
PH
duy nq phi 3 mini 4k merge v4
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
MI
yjwon mp mistral7b sft1 b512 lr5e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
BFS Search mistral multi rel I2B2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-2-7b-instruct
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GP
Gpt2 Large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
Llama-2-13b-chat-longlora-32k-sft
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
mlfoundations dev mistral alpaca sft sample 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CH
ChatAYT Lora Assamble Marcoroni
Chatayt · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MA
Magpie Align MagpieLM 8B SFT v0.1
Magpie · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
KO
KoSOLAR-10.7B-v0.2
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
LL
LLama 3.1 8B HyperNova
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
EL
Eleuther Pythia70M Hh Dpo
Eleuther · RichardErkhov
Params70M
Context2K
Q4_K_M0.0 GB
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GR
Iq Code Evmind v1 Granite 8b Instruct
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
LL
mlfoundations dev llama2 oh teknium scaling down random 0.5
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
ED
edbeeching EleutherAI pythia 2.8b
Edbeeching · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GP
Gpt2 Cia World Factbook
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
ZE
zephyr-beta-Nebula-v2-7B
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepFeatTextEmbeddingLarge
Deepfeattextembeddinglarge · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
ZE
zephyr-7b-dpo-full-judge-before
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 1B Instruct Zh De Block
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB