Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama3-ChatQA-1.5-8B
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
NX
Nxcode-CQ-7B-orpo
Nxcode · legraphista
Params7.25B
Context64K
Smallest2.4 GB
Coding
YI
Yi 1.5 34B Chat 16K
Yi · RichardErkhov
Params34.4B
Context16K
Q4_K_M20.7 GB
GGUF
LL
Llama-2-7b-WikiChat-fused
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
UN
Unlimited OCR 2
Unlimited · AtomicChat
Params2.94B
Context128K
Smallest2.0 GB
Vision
RE
realvisxl v5.0
Realvisxl · off-grid-ai
Params3.47B
Context—
Smallest2.8 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
RoleplayTool use
PO
pony diffusion v6 xl
Pony · off-grid-ai
Params3.47B
Context—
Smallest2.8 GB
GGUF
LL
Na0s Llama 3.1 8b Pruned 4 Layers
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
GGUF
TI
Tinyllama-Cinder-1.3B-Reason-Test
Tinyllama · RichardErkhov
Params1.28B
Context2K
Q4_K_M0.8 GB
GGUF
MI
Young Children Storyteller Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CO
confucius4 tts
Confucius · cstr
Params112M
Context—
Smallest0.1 GB
GGUF
SE
SenseVoiceSmall
Sensevoicesmall · cstr
Params234M
Context—
Smallest0.1 GB
GGUF
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
PA
parakeet tdt 0.6b ja
Parakeet · cstr
Params623M
Context—
Smallest0.4 GB
GGUF
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Q4_K_M4.5 GB
GGUF
PH
Unsloth Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Smallest0.9 GB
GGUF
OP
OpenELM-3B-Instruct
Openelm · SanctumAI
Params3.04B
Context2K
Q4_K_M1.9 GB
GGUF
GL
Glm Edge 4b Chat
GLM · zai-org
Params4.33B
Context3K
Q4_K_M2.6 GB
GGUF
LL
chargoddard llama 2 26b trenchcoat stack
Llama · RichardErkhov
Params25.7B
Context—
Q4_K_M5.5 GB
GGUF
SW
Swallow-70b-NVE-instruct-hf
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
IR
Irodori-TTS-v4.1-Small-MF
Irodori · sakasegawa
Params866M
Context—
Smallest0.9 GB
GGUF
FL
Flux2 Klein 9B True V1
Flux · wikeeyang
Params9.08B
Context—
Smallest5.7 GB
GGUF
TO
tokyotech llm Swallow 7b NVE instruct hf
Tokyotech · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
RO
RoGemma-7b-Instruct
Rogemma · legraphista
Params8.54B
Context8K
Smallest2.2 GB
GGUF
QW
Qwen2-7B
Qwen · legraphista
Params7.62B
Context128K
Smallest1.9 GB
GGUF
DE
DeepSeek-Coder-V2-Lite-Instruct
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
Coding
MI
Mistral 7B Instruct DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FL
FLUX.3 Image Klein
Flux · DogukanUrker
Params5.55B
Context—
Smallest2.5 GB
GGUF

Other quants

Keep browsing