Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
Gemma 3b 002
Gemma · RichardErkhov
Params3.17B
Context8K
Q4_K_M2.0 GB
GGUF
KA
Katayoon VPO SELM Zephyr 7B iter 2
Katayoon · RichardErkhov
Params7.24B
Context32K
Q4_K_M1.5 GB
GGUF
MI
Infinity Instruct 3M 0625 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EM
Embodied.cpp
Embodied · SEU-PAISys
Params1.3B
Context—
Smallest0.6 GB
GGUF
LL
Llama-3-8B-Instruct-v0.4
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
johnpaulbin llama3.1 8b tokipona toki translate 1 fp16
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MA
MagicPrompt-Dalle
Magicprompt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
PH
jasonhwan phi3 redteamer
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
MI
MiniLLM Gpt2 120M
Minillm · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
inswave AISquare Instruct llama2 koen 13b v0.9.5
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
MI
AlexeyRyzhikov mistral7b text to json v5.20
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
PH
fractalego wafl phi3 mini 4k
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
MI
mistral-7b-ft-h4-no_robots_instructions
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GL
Glm 4 9b
GLM · RichardErkhov
Params9.4B
Context8K
Q4_K_M6.3 GB
GGUF
TA
Taiwan LLM 13B v2.0 Chat
Taiwan · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
TO
toastloaf autotrain gpt2 finetune crab
Toastloaf · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2-135M-FT-SCP-Wiki
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange proofassistants
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
IM
P H3 Image
Image · AtomicChat
Params20.2B
Context—
Smallest9.0 GB
GGUF
GE
gemma-2b-translation-v0.150
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen2 7B Instruct Refine
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
AI
aisha partha medical question and answer gpt2
Aisha · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MA
Magnum Instruct DPO
Magnum · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
GE
Gemma-Wukong-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF

Other quants

Keep browsing