Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
UC
UCLA AGI zephyr 7b sft full SPIN iter0
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mrm8488 mistral 7b ft AgentInstruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
dolphin-2.9.1-mixtral-1x22b
Mistral · legraphista
Params22.2B
Context64K
Smallest4.8 GB
FA
Falcon H1 1.5B Instruct
Falcon · tiiuae
Params1.55B
Context128K
Q4_K_M0.9 GB
LL
dolphin-2.9.1-llama-3-70b
Llama · legraphista
Params70.6B
Context8K
Smallest15.3 GB
MI
internlm2-math-plus-mixtral8x22b
Mistral · legraphista
Params141B
Context64K
Smallest29.6 GB
MI
K2S3 Mistral 7bx2 48layers_v1.2
Mistral · RichardErkhov
Params10.9B
Context32K
Q4_K_M6.6 GB
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
AV
average-dolphin-8x7B
Average · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
TO
tokyotech llm Swallow 13b instruct hf
Tokyotech · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
NU
NuminaMath-7B-TIR
Numinamath · legraphista
Params6.91B
Context4K
Smallest1.7 GB
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Amory0201
Params7.5B
Context128K
Q4_K_M5.3 GB
MI
MiMo V2.5
Mimo · huihui-ai
Params310B
Context1024K
Smallest186.7 GB
LL
Llama 2 13b Hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
RW
rwkv 6 finch 1b6
Rwkv · latestissue
Params1.6B
Context1024K
Smallest0.7 GB
LL
Llama3 8b Cpt Sea Lionv2 Instruct
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3-ASR-0.6B
Qwen · sakasegawa
Params782M
Context—
Smallest0.5 GB
UC
UCLA AGI zephyr 7b sft full SPIN iter3
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
realtreetune deepseekmath 7b sft GSM8K
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GR
Granite 34b Code Base 8k
Granite · RichardErkhov
Params33.7B
Context8K
Q4_K_M21.4 GB
MI
K2S3 Mistral 7b v1.46
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
MO
MOSS Transcribe Diarize
Moss · cstr
Params909M
Context—
Smallest0.0 GB
PA
parakeet rnnt 0.6b
Parakeet · cstr
Params617M
Context—
Smallest0.4 GB
MI
Mistral-7B-v0.3
Mistral · legraphista
Params7.25B
Context32K
Smallest1.6 GB
LL
Llama 3 8B Instruct abliterated
Llama · failspy
Params8B
Context8K
Smallest4.9 GB
GE
Gemma 2 2b
Gemma · legraphista
Params2.61B
Context8K
Smallest0.8 GB
ST
stabilityai japanese stablelm instruct gamma 7b
Stabilityai · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
GI
Gille StrangeMerges 50 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB