Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Llama3-ChatQA-1.5-8B
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
NX
Nxcode-CQ-7B-orpo
Nxcode · legraphista
Params7.25B
Context64K
Smallest2.4 GB
YI
Yi 1.5 34B Chat 16K
Yi · RichardErkhov
Params34.4B
Context16K
Q4_K_M20.7 GB
LL
Llama-2-7b-WikiChat-fused
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
UN
Unlimited OCR 2
Unlimited · AtomicChat
Params2.94B
Context128K
Smallest2.0 GB
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Hermes 3 Llama 3.1 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
LL
Na0s Llama 3.1 8b Pruned 4 Layers
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
TI
Tinyllama-Cinder-1.3B-Reason-Test
Tinyllama · RichardErkhov
Params1.28B
Context2K
Q4_K_M0.8 GB
MI
Young Children Storyteller Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Q4_K_M4.5 GB
PH
Unsloth Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Smallest0.9 GB
OP
OpenELM-3B-Instruct
Openelm · SanctumAI
Params3.04B
Context2K
Q4_K_M1.9 GB
GL
Glm Edge 4b Chat
GLM · zai-org
Params4.33B
Context3K
Q4_K_M2.6 GB
LL
chargoddard llama 2 26b trenchcoat stack
Llama · RichardErkhov
Params25.7B
Context—
Q4_K_M5.5 GB
SW
Swallow-70b-NVE-instruct-hf
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
IR
Irodori-TTS-v4.1-Small-MF
Irodori · sakasegawa
Params866M
Context—
Smallest0.9 GB
FL
Flux2 Klein 9B True V1
Flux · wikeeyang
Params9.08B
Context—
Smallest5.7 GB
TO
tokyotech llm Swallow 7b NVE instruct hf
Tokyotech · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
RO
RoGemma-7b-Instruct
Rogemma · legraphista
Params8.54B
Context8K
Smallest2.2 GB
QW
Qwen2-7B
Qwen · legraphista
Params7.62B
Context128K
Smallest1.9 GB
DE
DeepSeek-Coder-V2-Lite-Instruct
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
MI
Mistral 7B Instruct DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-3-Instruct-8B-SPPO-Iter3
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
QW
Qwen2 Math 1.5B Instruct
Qwen · legraphista
Params1.54B
Context4K
Smallest0.4 GB
FL
PornMaster flux2 klein 9b
Flux · VenerableRolliPolli
Params8.19B
Context256K
Q4_K_M5.0 GB
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
IN
InternLM2
InternLM · legraphista
Params1.89B
Context8K
Smallest0.5 GB