Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
TL
tlphams solar 10.7b merged v0.1
Tlphams · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen3.5 9B
Qwen · titan0115
Params8.95B
Context256K
Q4_K_M5.6 GB
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
MA
MadKFC MergedCPsyModel
Madkfc · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
FA
Falcon H1 Tiny Coder 90M
Falcon · tiiuae
Params108M
Context256K
Q4_K_M0.1 GB
NO
notHumpback-Myx
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
LL
scandukuri llama3 8b stargate m1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma 2b It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
DI
distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context32K
Smallest1.9 GB
LE
Leia Swallow 7b
Leia · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
LL
uukuguy speechless codellama orca platypus 13b 0.10e
Llama · RichardErkhov
Params430M
Context—
Q4_K_M0.0 GB
TO
tomasonjo text2cypher demo 16bit
Tomasonjo · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LL
ZhangShenao SELM Llama 3 8B Instruct iter 1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
Mistral Nemo Instruct 2407
Mistral · legraphista
Params12.2B
Context1000K
Smallest3.0 GB
GI
Gille StrangeMerges 52 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
granite-7b-lab
Granite · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
OL
OLMo 7B 0424 hf Quantized
OLMo · aifoundry-org
Params6.89B
Context4K
Smallest1.6 GB
GE
Gemma 2 9B It SPPO Iter3
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
PR
Fish Speech S2 Pro Dual AR
Pro · subspecs
Params4.95B
Context32K
Q4_K_M3.6 GB
HE
0 hero Matter 0.1 7B
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
HE
0 hero Matter 0.1 Slim 7B B
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
llm-compiler-13b
Llm · legraphista
Params13B
Context16K
Smallest2.9 GB
QW
Qwen3.5 4B Base
Qwen · titan0115
Params4.21B
Context256K
Q4_K_M2.7 GB
LL
BatsResearch Llama 3.1 8B bonito v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
deepseek-coder-6.7b-base
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
PL
Plamo 2 Translate
Plamo · mitmul
Params9.53B
Context10240K
Q4_K_M6.3 GB
YI
Yi-1.5-9B-32K
Yi · RichardErkhov
Params8.83B
Context32K
Q4_K_M5.3 GB
KU
kukulemon-spiked-9B
Kukulemon · RichardErkhov
Params8.99B
Context8K
Q4_K_M5.4 GB