Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
BI
BioMistral BioMistral 7B DARE
Biomistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
QW
Canum med Qwen3 Reasoning
Qwen · prithivMLmods
Params1.72B
Context40K
Q4_K_M1.1 GB
LL
Llama3.2 1b Train Aware Quantization
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
NO
nota ai st vicuna v1.3 5.5b ppl
Nota · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
TE
Tensoic TinyLlama 1.1B 2.5T openhermes
Tensoic · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
OR
orpheus_3b_0.1_GGUF
Orpheus · Prince-1
Params3.3B
Context128K
Q4_K_M2.1 GB
RF
rfdetr cpp medium
Rfdetr · mudler
Params33M
Context—
Smallest0.0 GB
GR
granite 4.1 8b
Granite · eaddario
Params8.79B
Context128K
Smallest0.0 GB
PR
prometheus eval prometheus bgb 8x7b v2.0
Prometheus · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
XL
wav2vec2 xls r 300m uk with small lm
Xls · cstr
Params315M
Context—
Smallest0.2 GB
WH
whisper large v3
Whisper · oxide-lab
Params1.61B
Context—
Smallest0.5 GB
PH
Agnuxo Phi 3.5 Trinitron Instruct CODE Python English Asistant 16bit v2
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
OL
OLMoE 1B 7B 0924 Base
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
AR
ArmurAI Pentest AI
Armurai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
stt en fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
OM
OmniVoice Models Backup
Omnivoice · jhonpaz343445224
Params—
Context—
Q4_K_M0.3 GB
SE
seyf1elislam WestKunai Hermes 7b
Seyf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CL
Climategpt 7b Fsg
Climategpt · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B Instruct ORPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
LL
Llama 3.1 Storm 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TR
trlm 135m
Trlm · prithivMLmods
Params135M
Context8K
Q4_K_M0.1 GB
ST
stt kk ru fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
ST
stt ka fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
LL
4yo1 llama3 eng ko 8 llama
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
llm4decompile-1.3b
Llm · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GE
Gemma 2 9b
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
QW
JackChew Qwen2 VL 2B OCR
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB