Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
YI
Yi 34B 200K DARE Merge v7
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
FL
FLUX.1 Fill dev
Flux · second-state
Params11.9B
Context—
Smallest4.2 GB
LL
PiSSA Llama 2 7b R128
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
OP
Openbuddy Yi1.5 34b v21.3 32k
Openbuddy · RichardErkhov
Params34.4B
Context128K
Q4_K_M20.7 GB
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
MI
NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
CL
ClaudioItaly Evolutionstory 7B v3.0
Claudioitaly · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
YuisekinAI Mistral 0.7B
Mistral · RichardErkhov
Params746M
Context4K
Q4_K_M0.5 GB
MI
flammenai flammen15X mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 0.5B Instruct
Qwen · legraphista
Params494M
Context32K
Smallest0.3 GB
MI
speechless-mistral-six-in-one-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
KU
Kukedlc NeuralSynthesis 7B v0.3
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AT
Athene 70B
Athene · legraphista
Params70.6B
Context8K
Smallest15.3 GB
UM
Umievo Itr012 Gleipnir 7B
Umievo · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
QW
gte-Qwen2-1.5B-instruct
Qwen · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
ST
StableBeluga 7B
Stablebeluga · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
SU
SurgeGlobal OpenBezoar HH RLHF SFT
Surgeglobal · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
PH
phi 2 layla v1 chatml
Phi · l3utterfly
Params2.78B
Context2K
Q4_K_M1.7 GB
QW
Qwen_Qwen3 1.7B
Qwen · Alcoft
Params2.03B
Context40K
Q4_K_M1.3 GB
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
YI
Yi 1.5 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
MI
Kant-Test-0.1-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
Starling LM 7B Beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
VE
VertaLily 1.2 1B
Vertalily · VLTX
Params1.17B
Context125K
Q4_K_M0.7 GB
MI
microsoft_WizardLM-2-7B
Microsoft · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
MI
Mistral 7b Sandbagging Cb New
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB