Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
WH
whisper base
Whisper · oxide-lab
Params73M
Context—
Smallest0.0 GB
LL
Llama 3.2 Taiwan 3B Instruct
Llama · lianghsun
Params3.61B
Context128K
Q4_K_M2.2 GB
GR
Granite 20b Code Base 8k
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
LL
neuralmagic Llama 2 7b evolcodealpaca
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
CA
Calme-7B-Instruct-v0.9
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PS
Psyfighter2 13B Vore
Psyfighter · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
LL
llm-compiler-7b-ftd
Llm · legraphista
Params6.74B
Context16K
Smallest1.5 GB
OP
OpenELM-1_1B-Instruct
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
QW
Qwen2-57B-A14B-Instruct
Qwen · legraphista
Params57.4B
Context32K
Smallest21.1 GB
SA
Salt Asr_Wav Uni_1_Tts_Wav Uni_1 12k
Salt · RichardErkhov
Params3.23B
Context128K
Q4_K_M2.0 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · ahmed-amedo
Params1.86B
Context—
Smallest0.9 GB
GR
Granite 8b Code Instruct 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
MI
mistralai Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama3 8b Math Sft
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
DE
DeepSeek-Coder-V2-Lite-Base
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
LL
Meta Llama 3.1 70B Instruct
Llama · legraphista
Params70.6B
Context128K
Smallest15.3 GB
QW
Qwen2.5 14B Instruct
Qwen · legraphista
Params14.8B
Context32K
Smallest3.6 GB
OC
octo net
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
BO
Bookworm 10.7B v0.4 DPO
Bookworm · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
MI
flammenai flammen20 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
FL
Flux2 Klein 9B True V3
Flux · sanrenti
Params9.08B
Context—
Smallest5.7 GB
OP
OpenELM-450M-Instruct
Openelm · RichardErkhov
Params457M
Context2K
Q4_K_M0.3 GB
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · li1272
Params8.2B
Context256K
Smallest3.3 GB
MO
moss transcribe.cpp
Moss · mudler
Params909M
Context—
Smallest0.5 GB
LL
Llama 3.2 1B Instruct Abliterated
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
MI
Mistral 7B ReMax v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QV
QVikhr 3 8B Instruction
Qvikhr · Vikhrmodels
Params8.19B
Context40K
Q4_K_M5.0 GB
LL
kykim0 Llama 2 7b ultrachat code
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB