Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
RE
realvisxl v5.0 lightning
Realvisxl · off-grid-ai
Params3.47B
Context—
Smallest2.8 GB
GGUF
TI
TinySolar-248m-4k-py
Tinysolar · RichardErkhov
Params248M
Context32K
Q4_K_M0.2 GB
GGUF
FI
FireRedASR2-AED
Fireredasr · cstr
Params1.18B
Context—
Smallest1.0 GB
GGUF
QW
Qwen3-TTS-12Hz-0.6B-CustomVoice
Qwen · sakasegawa
Params1.02B
Context—
Smallest0.8 GB
GGUF
QW
qwen3 asr 1.7b ja anime
Qwen · cstr
Params2.35B
Context—
Smallest1.5 GB
GGUF
CO
ConfigurableBeagle-11B
Configurablebeagle · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
YI
Yi 1.5 34B 32K
Yi · RichardErkhov
Params34.4B
Context32K
Q4_K_M20.7 GB
GGUF
MI
speechless-mistral-dolphin-orca-platypus-samantha-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AL
allknowingroger CalmExperiment 7B slerp
Allknowingroger · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
JP
jpacifico Chocolatine 14B Instruct DPO v1.2
Jpacifico · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
LL
hon9kon9ize Cantonese Llama 2 7B preview20240625
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · SanctumAI
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
GR
granite speech 4.1 2b
Granite · cstr
Params2.31B
Context—
Smallest1.8 GB
GGUF
YI
Yi 34B 200K DARE Merge v7
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
FL
FLUX.1 Fill dev
Flux · second-state
Params11.9B
Context—
Smallest4.2 GB
GGUF
LL
PiSSA Llama 2 7b R128
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
FishAudio Drama3 TTS 83Lang Qwen3.5 35B A3B FP8 BF16
Qwen · AMAImedia
Params35B
Context—
Smallest23.4 GB
GGUF
OP
Openbuddy Yi1.5 34b v21.3 32k
Openbuddy · RichardErkhov
Params34.4B
Context128K
Q4_K_M20.7 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
FL
PornMaster flux2 klein 9b
Flux · cl0537furry
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
CL
ClaudioItaly Evolutionstory 7B v3.0
Claudioitaly · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
YuisekinAI Mistral 0.7B
Mistral · RichardErkhov
Params746M
Context4K
Q4_K_M0.5 GB
GGUF
MI
flammenai flammen15X mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 0.5B Instruct
Qwen · legraphista
Params494M
Context32K
Smallest0.3 GB
GGUF
MI
speechless-mistral-six-in-one-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KU
Kukedlc NeuralSynthesis 7B v0.3
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AT
Athene 70B
Athene · legraphista
Params70.6B
Context8K
Smallest15.3 GB
GGUF
UM
Umievo Itr012 Gleipnir 7B
Umievo · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing