Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
NO
nothingiisreal MN 12B Starcannon v3
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
scb10x llama 3 typhoon v1.5x 70b instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
LLaMA 3 8B SFR Iterative DPO R
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
FL
PornMaster Klein 9b
Flux · fatangel420
Params8.19B
Context256K
Q4_K_M5.0 GB
VI
vicgalle ConfigurableHermes 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Smallest0.8 GB
DE
DeepSeek-OCR
DeepSeek · NexaAI
Params2.93B
Context—
Smallest1.7 GB
LL
Llama 3 Instruct 8B SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
DE
Husky v1 Math Deepseekmath 7b Instruct
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
LL
chinese llama 2 7b
Llama · hfl
Params6.93B
Context4K
Smallest2.6 GB
MI
nbeerbower bophades mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RA
RakutenAI 7B chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
timestral cpp
Mistral · mudler
Params123M
Context32K
Smallest0.0 GB
CH
ChaoticNeutrals Prima LelantaclesV5 7b
Chaoticneutrals · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IR
irodori tts voicedesign
Irodori · cstr
Params71M
Context—
Smallest0.1 GB
LL
Llama 3 Swallow 70B v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
SP
SparseLlama 3 8B Pruned_50.2Of4
Sparsellama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
PA
parler tts mini v1.1
Parler · cstr
Params—
Context—
Smallest0.0 GB
PO
Poro 34B
Poro · RichardErkhov
Params34B
Context7K
Q4_K_M22.4 GB
QW
Qwen2 VL 2B Instruct
Qwen · turingevo
Params1.54B
Context32K
Q4_K_M1.0 GB
SM
SmallThinker 21BA3B Instruct
Smallthinker · Tiiny
Params21.5B
Context16K
Q4_K_M13.1 GB
SM
gguf SmolLM 135M Instruct
SmolLM · Felladrin
Params135M
Context2K
Smallest0.1 GB
PH
Phi 3.5 Mini Instruct
Phi · SanctumAI
Params3.82B
Context128K
Q4_K_M2.4 GB
ME
MetaMath Bagel DPO 34B
Metamath · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
MO
MobileVLM V2 1.7B
Mobilevlm · ZiangWu
Params1.36B
Context2K
Smallest0.8 GB
HA
Haas-ja-v1.0
Haas · sakasegawa
Params616M
Context—
Smallest0.4 GB
OL
OLMo 7B Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
VI
vinallama-2.7b
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB