Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
dolphin-2.1-mistral-7b-snr-math-laser
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DI
distill_gpt2_story_generator
Distill · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
DI
Distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · minte1431
Params1.86B
Context256K
Smallest0.9 GB
LL
Pair Preference Model LLaMA3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
PY
Pythia-160M-Deduped-Adventure
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
MI
OpenMath-Mistral-7B-v0.1-hf
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AI
aialt MMedL3 MI
Aialt · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
OR
it-5.4-fp16-orpo-v2
Orpo · Vikhrmodels
Params7.63B
Context32K
Q4_K_M4.6 GB
MO
Monero WizardLM 30B Uncensored Guanaco SuperCOT 30b
Monero · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
FA
Falcon
Falcon · RichardErkhov
Params7.22B
Context2K
Q4_K_M5.0 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
TR
trojblue gpt2 prompt upscaler v1
Trojblue · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
ST
starcoder2-3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
QW
online-dpo-qwen2-2
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
FA
Falcon
Falcon · RichardErkhov
Params11.1B
Context2K
Q4_K_M6.8 GB
ET
ethzanalytics ai msgbot gpt2 XL dialogue
Ethzanalytics · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
LL
rasa cmd gen codellama 13b calm demo
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
LL
CodeLlama 13b Instruct Hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
MI
Bitnet-Mistral.0.2-330m-v0.2-grokfast-v2.9
Mistral · RichardErkhov
Params330M
Context4K
Q4_K_M0.2 GB
OL
OLMoE 1B 7B 0924 Instruct Base
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
SM
SmolLM2-135M-Instruct-K12-5000
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
IN
interview eval zephyr 7b math case 8 fixed
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral Instruct Mcqa Circuit Broken
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 3.0 3b A800M Base
Granite · RichardErkhov
Params3.37B
Context4K
Q4_K_M2.1 GB
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b2.0
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB