Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
GP
Gpt2 Cia World Factbook
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
ZE
zephyr-beta-Nebula-v2-7B
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepFeatTextEmbeddingLarge
Deepfeattextembeddinglarge · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
ZE
zephyr-7b-dpo-full-judge-before
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BC
Bce Embedding Base_V1
Bce · gpustack
Params277M
Context512
Q4_K_M0.2 GB
LL
Llama 3.2 1B Instruct Zh De Block
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
QW
psneto qwen2 arithmetic 2T FT
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
IN
interview eval zephyr 7b stem case 7
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
DE
DeepSeek V4 Flash 0731
DeepSeek · tarruda
Params284B
Context1024K
Smallest0.5 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
LL
verbalyze Llama3.1 finetuned
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
RE
reverie-7b
Reverie · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
ministral Ministral 3b instruct
Mistral · RichardErkhov
Params3.32B
Context128K
Q4_K_M2.0 GB
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo camel biology gpt 4o mini
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
CodeLlama 34B Python Fp16
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
ML
mlfoundations dev oh teknium scaling down random 0.1
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama Guard 3 1B
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
DB
Dbrx Instruct
Dbrx · RichardErkhov
Params208B
Context32K
Q4_K_M79.9 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
TE
Teuken 7B Base v0.6
Teuken · shahzeb171
Params8.48B
Context4K
Smallest3.8 GB
SA
SambaLingo Russian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GP
gpt2-medium-italian-embeddings
Gpt · RichardErkhov
Params365M
Context1K
Q4_K_M0.2 GB