Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
XA
XALMA 13B Pretrain
Xalma · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
YI
HenryJJ Instruct Yi 6B Dolly CodeAlpaca
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
SM
SmolLM2 135M Instruct
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Vision
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · datamart99
Params1.86B
Context—
Smallest0.9 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
grounded ai phi3 rag relevance judge merge
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
collaiborateorg Collaiborator MEDLLM Llama 3 8B v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PA
parakeet-tdt-0.6b-v3
Parakeet · sakasegawa
Params627M
Context—
Smallest0.4 GB
GGUF
OL
ai2 adapt dev olmo 7b peteish7 step928646
OLMo · RichardErkhov
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
FairMind Minerva 3B Instruct v1.0
Fairmind · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Llama 3 8B Instruct SPPO Score Iter2_gp_8b Table 0.002
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
FO
ppformulanet-l-ocr
Formulanet · cstr
Params181M
Context—
Smallest0.1 GB
GGUF
LL
hirushanan Code LLaMA 7b optimize code
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 3 10b Wiki 240709 F
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
GGUF
QW
Qwen1.5-1.8B-Chat
Qwen · RichardErkhov
Params1.84B
Context32K
Q4_K_M1.2 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
suzume-llama-3-8B-multilingual-orpo-borda-top25
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
LL
Youngwon Llama 3.2 1B info 4bit
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
Dolphin 2.9 Llama3 8b 256k
Llama · RichardErkhov
Params8.03B
Context512K
Q4_K_M4.9 GB
GGUF
LL
haqishen Llama 3 8B Japanese Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo camel ai chemistry
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Prosparse Llama 2 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
PH
Phi 2 Electrical Engineering
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF

Other quants

Keep browsing