Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
WA
wav2vec2 large xlsr 53 arabic
Wav · cstr
Params315M
Context—
Smallest0.2 GB
QW
RESMPDEV Qwen2 Wukong 5B Instruct Sliced
Qwen · RichardErkhov
Params5.75B
Context32K
Q4_K_M3.6 GB
AG
AgentEvol 7B
Agentevol · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
OL
OLMo 1B 0724 Hf
OLMo · aifoundry-org
Params1.28B
Context4K
Smallest0.3 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP GGUF Q4 K P
Qwen · Vincentt
Params27.3B
Context256K
Smallest17.9 GB
PO
Poro 34B Chat
Poro · RichardErkhov
Params35.1B
Context7K
Q4_K_M22.4 GB
GE
thusinh1969 gemma2 2b rerank checkpoint 8800
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
LL
scb10x llama 3 typhoon v1.5x 8b instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
TheDrummer Big Tiger Gemma 27B v1
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
LL
Laim Llama 3.1 MedPalm2 imitate 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
AL
alpindale pygmalion instruct
Alpindale · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
CH
CheckGPT SOLAR 10.7B
Checkgpt · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GE
gemma 4 12B it qat uncensored heretic UDmerge
Gemma · EZForever
Params11.9B
Context256K
Smallest6.7 GB
NE
IQ2 NL IQ3 NL Test Models
Nemotron · eaddario
Params8.89B
Context1024K
Smallest0.0 GB
MO
MOSS Transcribe preview 2B
Moss · cstr
Params2.42B
Context—
Smallest0.0 GB
LL
Llama 3 6B v0
Llama · RichardErkhov
Params6.29B
Context8K
Q4_K_M3.9 GB
PY
Pyxidis Manim CodeGen 1.7B
Pyxidis · prithivMLmods
Params1.72B
Context40K
Q4_K_M1.1 GB
PH
Phi 3.5 Mini instruct_Uncensored
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
RF
rfdetr cpp seg nano
Rfdetr · mudler
Params33M
Context—
Smallest0.0 GB
PH
Phi 3 medium 128k instruct IMat
Phi · legraphista
Params14B
Context128K
Smallest3.0 GB
LE
Leia Swallow 13b
Leia · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
AN
anthracite org magnum v2 32b
Anthracite · RichardErkhov
Params32.5B
Context32K
Q4_K_M19.7 GB
QW
MAmmoTH2 8B Plus
Qwen · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
OP
OpenLLM Ro RoLlama2 7b Instruct 2024 05 14
Openllm · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B v0.1 Half Naive A
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
finetune-Qwen2-0.5B-codeDataset
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
SH
Shieldgemma 9b
Shieldgemma · legraphista
Params9.24B
Context8K
Smallest2.4 GB
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
llama-3-experiment-v1-9B
Llama · RichardErkhov
Params8.9B
Context8K
Q4_K_M5.4 GB
DE
DeepSeek V2 Lite
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB