Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PA
ekko v1 tiny
Parakeet · RyeAI
Params113M
Context—
Smallest0.1 GB
GGUF
QW
ReAligned Qwen3.5 35B A3B
Qwen · Lazarus-Ai
Params34.7B
Context256K
Smallest12.8 GB
ReasoningTool use
GE
1 800 SHARED TASKS GEMMA2 27B NLI 16bit
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
CH
chinese alpaca 2 1.3b
Chinese · hfl
Params1.26B
Context4K
Smallest0.6 GB
GGUF
DE
Deepseek Math 7b Rl
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
MI
Mistral NeMo Minitron 8B Base
Mistral · RichardErkhov
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
MO
moonshine-base
Moonshine · cstr
Params62M
Context—
Smallest0.0 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · htrhdjy
Params1.86B
Context—
Smallest0.9 GB
GGUF
LI
lightblue reranker 0.5 cont
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use
CU
CultriX NeuralCeptrix 7B SLERP
Cultrix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
StructLM 7B Mistral
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
NousResearch Hermes 2 Pro Llama 3 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
LL
Llama3-70B-Fireplace
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
QW
INFRL Qwen2.5 VL 72B Preview ggufs fully quantized
Qwen · GeorgyGUF
Params72.7B
Context125K
Q4_K_M47.4 GB
Tool useVision
MI
miotts 0.6b
Miocodec · cstr
Params600M
Context—
Smallest0.0 GB
GGUF
AR
aragpt2-base
Aragpt · RichardErkhov
Params184M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Mistral Large Instruct 2407
Mistral · RichardErkhov
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · xbann
Params8.95B
Context256K
Smallest3.6 GB
ReasoningRoleplayTool use
LL
dakshvar22 cmd gen travel assistant codellama 7b unsloth params
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange reverseengineering
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2 Math 7B Instruct
Qwen · legraphista
Params7.62B
Context4K
Smallest1.9 GB
GGUF
LL
dfurman Llama 3 8B Orpo v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma-4-E2B-Uncensored-HauhauCS-Aggressive
Gemma · kamjin
Params4.65B
Context128K
Smallest3.0 GB
ReasoningTool useVision
DH
Dhana8907 labsmergedModel0312
Dhana · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
tchen175 llama3.1 8b per sent
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
gemma 4 E2B it
Gemma · eaddario
Params4.65B
Context128K
Smallest0.0 GB
GGUF
LL
Taigi Llama 2 Translator 13B
Llama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
GGUF
LA
lamm mit PRefLexOR ORPO DPO EXO 10242024
Lamm · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use
HO
Hojo ASR V1
Hojo · cstr
Params4.81B
Context—
Smallest4.4 GB
GGUF
LL
aifeifei798 llama3 8B DarkIdol 2.1 Uncensored 32K
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing