Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
BR
Breeze-7B-Base-v1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
WA
wav2vec2 large xlsr 53 portuguese
Wav · cstr
Params315M
Context—
Smallest0.2 GB
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
BI
Bifrost-27B-GGUF
Bifrost · Joumdane
Params27B
Context128K
Q4_K_M16.5 GB
ST
stt fr fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · KilomAkel
Params34.7B
Context256K
Q4_K_M21.2 GB
SE
sense voice
Sense · lovemefan
Params234M
Context—
Smallest0.2 GB
QW
Josiefied Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GL
GLM 4.6V Flash
GLM · eaddario
Params9.4B
Context128K
Q4_K_M6.2 GB
GE
GeM2 Llamion 14B Base
Gem · RichardErkhov
Params14.5B
Context4K
Q4_K_M8.8 GB
AU
AutoCoder_S_6.7B
Autocoder · legraphista
Params6.74B
Context16K
Smallest1.5 GB
YI
Yi-1.5-9B-Chat-16K
Yi · RichardErkhov
Params8.83B
Context16K
Q4_K_M5.3 GB
AR
archangel_sft_pythia2-8b
Archangel · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
QW
Qwen2.5 14B Instruct Abliterated
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
QW
Qwen2.5 Coder 1.5B Instruct Abliterated
Qwen · URWI08RUS
Params1.78B
Context32K
Q4_K_M1.1 GB
LL
llama 3 8b instruct 262k chinese
Llama · Monor
Params8.03B
Context256K
Q4_K_M4.9 GB
KA
kalisai Nusantara 2.7b Indo Chat
Kalisai · RichardErkhov
Params2.65B
Context8K
Q4_K_M1.7 GB
FL
Flan T5 Xxl
Flan · dumb-dev
Params11.1B
Context512
Q4_K_M6.8 GB
LL
AIML GEEK advanced promptv2 finetuned CodeLlama7b Instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
PH
Phi 3 medium 128k instruct
Phi · RDson
Params14B
Context128K
Q4_K_M8.6 GB
GE
google gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
JU
juggernaut xl v9 q4k
Juggernaut · MomoSoft
Params3.47B
Context—
Smallest3.0 GB
GP
gpt2-xl
Gpt · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
LI
lightblue reranker 0.5 bin filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
DO
DongfuJiang prm version3 full hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
YI
Yi 6b 200k
Yi · RichardErkhov
Params6.06B
Context195K
Q4_K_M3.7 GB
LL
MaziyarPanahi calme 2.2 llama3 70b
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
SA
Sailor 4B
Sailor · RichardErkhov
Params3.95B
Context32K
Q4_K_M2.5 GB
MI
mistralai Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RF
rfdetr cpp seg xlarge
Rfdetr · mudler
Params37M
Context—
Smallest0.0 GB