Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Magpie Align Llama 3 8B Magpie Align v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
AmpleGCG Llama2 Sourced Llama2 7b Chat
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
MI
Mistral-7B-Instruct-v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
OpenCodeInterpreter SC2 3B
Opencodeinterpreter · RichardErkhov
Params3.18B
Context16K
Q4_K_M2.0 GB
GGUF
MI
Ja ck Mistral instruct Y24 v5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
self-correct_Llama-3.2-3B-Instruct_metaMathQA_dpo_iter3
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use
PH
sujithatz finbot phi3.5 merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
QW
Qwen3.8 Flash Next GSQ RCO IQ2 MoREL PB3 304
Qwen · oisee
Params128B
Context256K
Smallest3.0 GB
CodingReasoningTool use
MI
avemio digital mistral continued training pre trained
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen Encoder 0.5B
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
Coding
QW
Qwen3.5 2B Heretic
Qwen · filvyb
Params1.88B
Context256K
Smallest1.3 GB
CodingReasoningTool use
OP
Open Instruct Stanford Alpaca 7b
Open · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
TR
trl internal testing tiny BloomForCausalLM
Trl · RichardErkhov
Params22M
Context—
Q4_K_M0.0 GB
GGUF
DO
DongfuJiang prm version3 subsample hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
yujiepan QwQ tiny random
Qwen · RichardErkhov
Params11M
Context—
Q4_K_M0.0 GB
Reasoning
LI
lightblue reranker 0.5 cont filt 7max
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use
MI
mlfoundations dev hp ablations mistral bsz256 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HE
Hermes Instruct 7B 217K
Hermes · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
CO
Yi Coder 1.5B Chat
Coder · RichardErkhov
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
QW
Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
QW
Qwen3.6 35B A3B Uncensored Wasserstein
Qwen · wazimondo
Params34.7B
Context256K
Smallest23.4 GB
ReasoningTool useVision
TI
TinyLlama-1.1B-Chat-v0.5
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B Tele It
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama TinyLlama 1.1B Chat v0.3
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SE
seyf1elislam WestKunai Hermes 10.7b test
Seyf · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
Tool use
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
MI
Mistral-11B-OmniMix
Mistral · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
OL
OLMo-1B-hf
OLMo · RichardErkhov
Params1.18B
Context2K
Q4_K_M0.7 GB
GGUF

Other quants

Keep browsing