Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PA
PaddleOCR-VL-0.9B
Paddleocr · cstr
Params943M
Context—
Smallest1.3 GB
GGUF
DE
Deepseek Coder 6.7B Kexer
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
mlfoundations dev hp ablations mistral scheduler linear warmup0.05
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Base SFT RDPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Mikhil jivus Llama 3.2 1B FineTuned
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Llama 3 Instruct 8B CPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ME
MexIvanov MistRAG 7B ruen v1 merged
Mexivanov · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
MI
avemio digital mistral 7b Sft orpo 3000 steps merge
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZE
Zenith 7B Dpo
Zenith · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
codellama CodeLlama 13b Instruct hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
QW
ReAligned Qwen3.5 4B
Qwen · Lazarus-Ai
Params4.21B
Context256K
Smallest1.8 GB
ReasoningTool use
DA
darkc0de BuddyGlass v0.3 Xortron7MethedUpSwitchedUp
Darkc · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
MinerU2.5-Pro-2604-1.2B
Mineru · cstr
Params1.16B
Context—
Smallest0.8 GB
GGUF
LL
Tiance llama 2 7b chat finetune
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · dahara1
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
QW
Qwen2.5 7b Instruct Cpt v1.1
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
MI
dolphin-2.6-mistral-7b-dpo-orca
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
hydra project ChatHercules 2.5 Mistral 7B
Mistral · RichardErkhov
Params470M
Context—
Q4_K_M0.3 GB
GGUF
MI
Mistral 7B Base SFT RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AM
ambrosfitz TinyLlama 1.1B Chat yawp
Ambrosfitz · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SM
SmolLM2 135M
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Coding
CO
CodeRosa 70B AB1
Coderosa · altomek
Params69B
Context4K
Smallest14.5 GB
Coding
KU
Kukedlc Neural Krishna Multiverse 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 8B Instruct Gradient 1048k
Llama · Monor
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen1.5-0.5B-Chat
Qwen · RichardErkhov
Params620M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 400 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF

Other quants

Keep browsing