Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Emma 500 Llama2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LO
louisbrulenaudet Maxine 7B 0401 stock
Louisbrulenaudet · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DR
DreadPoor Harpy 7B Model Stock
Dreadpoor · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
UC
UCLA AGI zephyr 7b sft full SPIN iter2
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NI
NicholasCorrado zephyr 7b uf rlced conifer dpo 2e
Nicholascorrado · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Minitron 4B Base
Minitron · legraphista
Params4.19B
Context4K
Smallest1.9 GB
LL
cognitivecomputations dolphin 2.9 llama3 8b
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
MI
Minimax H3 pruned
Minimax · PurpleBlazey
Params20.1B
Context256K
Smallest6.3 GB
VI
vicgalle Configurable Janus 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 14B Instruct Abliterated v2
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
MI
failspy Codestral 22B v0.1 abliterated v3
Mistral · RichardErkhov
Params22.2B
Context32K
Q4_K_M13.3 GB
MI
AmeerH Mistral Math 2x7b mix
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GE
Gemma 2 27b It Abliterated
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
WH
whisper base
Whisper · oxide-lab
Params73M
Context—
Smallest0.0 GB
LL
Llama 3.2 Taiwan 3B Instruct
Llama · lianghsun
Params3.61B
Context128K
Q4_K_M2.2 GB
GR
Granite 20b Code Base 8k
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
LL
neuralmagic Llama 2 7b evolcodealpaca
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
CA
Calme-7B-Instruct-v0.9
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GL
glm ocr crispembed
GLM · cstr
Params1.11B
Context—
Smallest0.0 GB
PS
Psyfighter2 13B Vore
Psyfighter · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
LL
llm-compiler-7b-ftd
Llm · legraphista
Params6.74B
Context16K
Smallest1.5 GB
GI
gigaam v3
Gigaam · cstr
Params221M
Context—
Smallest0.2 GB
OP
OpenELM-1_1B-Instruct
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
QW
Qwen2-57B-A14B-Instruct
Qwen · legraphista
Params57.4B
Context32K
Smallest21.1 GB
SA
Salt Asr_Wav Uni_1_Tts_Wav Uni_1 12k
Salt · RichardErkhov
Params3.23B
Context128K
Q4_K_M2.0 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · ahmed-amedo
Params1.86B
Context—
Smallest0.9 GB
MI
Mini-Omni2
Mini · cstr
Params782M
Context—
Smallest1.1 GB
GR
Granite 8b Code Instruct 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
MI
mistralai Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB