Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CO
cohere asr ja
Cohere · CKHO
Params2.07B
Context—
Smallest1.5 GB
GGUF
OM
OmniASR-LLM-300M
Omniasr · cstr
Params1.63B
Context—
Smallest1.1 GB
GGUF
LL
Llama3 8b Math Sft
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
DE
DeepSeek-Coder-V2-Lite-Base
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
Coding
LL
Meta Llama 3.1 70B Instruct
Llama · legraphista
Params70.6B
Context128K
Smallest15.3 GB
GGUF
PA
parakeet tdt 1.1b
Parakeet · cstr
Params1.07B
Context—
Smallest0.7 GB
GGUF
QW
Qwen2.5 14B Instruct
Qwen · legraphista
Params14.8B
Context32K
Smallest3.6 GB
GGUF
OC
octo net
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
BO
Bookworm 10.7B v0.4 DPO
Bookworm · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
MI
flammenai flammen20 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FL
Flux2 Klein 9B True V3
Flux · sanrenti
Params9.08B
Context—
Smallest5.7 GB
GGUF
OP
OpenELM-450M-Instruct
Openelm · RichardErkhov
Params457M
Context2K
Q4_K_M0.3 GB
GGUF
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · li1272
Params8.2B
Context256K
Smallest3.3 GB
CodingVision
MO
moss transcribe.cpp
Moss · mudler
Params909M
Context—
Smallest0.5 GB
GGUF
DA
data2vec audio 960h
Data · cstr
Params94M
Context—
Smallest0.1 GB
GGUF
LL
Llama 3.2 1B Instruct Abliterated
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GGUF
MI
Mistral 7B ReMax v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QV
QVikhr 3 8B Instruction
Qvikhr · Vikhrmodels
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
TE
text super resolution
Text · cstr
Params—
Context—
Smallest0.0 GB
GGUF
LL
kykim0 Llama 2 7b ultrachat code
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
Coding
QW
Qwen2.5 1.5B Instruct
Qwen · legraphista
Params1.54B
Context32K
Smallest0.4 GB
GGUF
LL
Llama 2 7b Ultrachat
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Higgs-Llama-3-70B
Llama · legraphista
Params70.6B
Context8K
Smallest15.3 GB
GGUF
HU
hubert large ls960 ft
Hubert · cstr
Params315M
Context—
Smallest0.2 GB
GGUF
LL
picAIso TARS 8B llama REMIX v2
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QU
QuantumIntelligence QI neural chat 7B ko DPO
Quantumintelligence · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
nemotron 3.5 asr streaming
Nemotron · cstr
Params638M
Context—
Smallest0.4 GB
GGUF
QW
Tifa 7B Qwen2 v0.1
Qwen · Tifa-RP
Params7.62B
Context32K
Q4_K_M4.7 GB
Roleplay
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing