Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
Phi 3.5 Mini instruct_Uncensored
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
PH
Phi 3 medium 128k instruct IMat
Phi · legraphista
Params14B
Context128K
Smallest3.0 GB
GGUF
LE
Leia Swallow 13b
Leia · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF
AN
anthracite org magnum v2 32b
Anthracite · RichardErkhov
Params32.5B
Context32K
Q4_K_M19.7 GB
GGUF
QW
MAmmoTH2 8B Plus
Qwen · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Reasoning
OP
OpenLLM Ro RoLlama2 7b Instruct 2024 05 14
Openllm · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B v0.1 Half Naive A
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
finetune-Qwen2-0.5B-codeDataset
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
SH
Shieldgemma 9b
Shieldgemma · legraphista
Params9.24B
Context8K
Smallest2.4 GB
GGUF
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
llama-3-experiment-v1-9B
Llama · RichardErkhov
Params8.9B
Context8K
Q4_K_M5.4 GB
GGUF
DE
DeepSeek V2 Lite
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
GGUF
PA
ekko v1 tiny
Parakeet · RyeAI
Params113M
Context—
Smallest0.1 GB
GGUF
QW
ReAligned Qwen3.5 35B A3B
Qwen · Lazarus-Ai
Params34.7B
Context256K
Smallest12.8 GB
ReasoningTool use
GE
1 800 SHARED TASKS GEMMA2 27B NLI 16bit
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
CH
chinese alpaca 2 1.3b
Chinese · hfl
Params1.26B
Context4K
Smallest0.6 GB
GGUF
DE
Deepseek Math 7b Rl
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
MI
Mistral NeMo Minitron 8B Base
Mistral · RichardErkhov
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · htrhdjy
Params1.86B
Context—
Smallest0.9 GB
GGUF
LI
lightblue reranker 0.5 cont
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
EmbeddingTool use
CU
CultriX NeuralCeptrix 7B SLERP
Cultrix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
StructLM 7B Mistral
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
NousResearch Hermes 2 Pro Llama 3 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
LL
Llama3-70B-Fireplace
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
AR
aragpt2-base
Aragpt · RichardErkhov
Params184M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Mistral Large Instruct 2407
Mistral · RichardErkhov
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
QW
Qwen3.5 9B Claude 4.6 HighIQ THINKING HERETIC UNCENSORED
Qwen · xbann
Params8.95B
Context256K
Smallest3.6 GB
ReasoningRoleplayTool use
LL
dakshvar22 cmd gen travel assistant codellama 7b unsloth params
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange reverseengineering
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2 Math 7B Instruct
Qwen · legraphista
Params7.62B
Context4K
Smallest1.9 GB
GGUF

Other quants

Keep browsing