Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mistral7B PairRM SPPO Iter1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RA
RaiBP gpt2 openwebtext2 first 30 chunks ablation bilingual
Raibp · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
JP
jpacifico French Alpaca 7B Instruct beta
Jpacifico · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HE
0 hero Matter 0.1 Slim 7B preview
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Lumimaid 8b v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5-0.5B-SFT-2e-5-2ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
XL
xLAM 7b Fc R
Xlam · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
Tool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
Karen_TheEditor_V2_STRICT_Mistral_7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
jacson llama 2 7b codewello
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
Llama 2 Koen 13b
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
LL
llama 2 13b langchain chat
Llama · shaowenchen
Params13B
Context2K
Smallest5.4 GB
GGUF
LL
Llama3 TAIDE LX 8B Chat Alpha1
Llama · nctu6
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
RO
ronigold dictalm2.0 instruct fine tuned alpaca gpt4 hebrew
Ronigold · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
DE
Srijith rkr deepseek base 1e 3 NO cot only failed samples 100 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
LE
LeetCodeWizard 7B V1.1
Leetcodewizard · Nan-Do
Params6.74B
Context16K
Smallest3.3 GB
Coding
YI
Yi 34b 200k Aezakmi Raw Toxic 2702
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
DU
duliadotio dulia 13b 8k alpha
Duliadotio · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
GGUF
MI
klcsp mistral 7b 0.3 gpt4o 100k classification fft
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
AT
Atlas Chat 9B
Atlas · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
GE
Gemma 2b Ru
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
DE
deepseek-coder-1.3b-python-peft
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
IG
Ignis 7B DPO
Ignis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
microsoft_WizardLM-2-7B
Microsoft · thesven
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma2 Nephilim v3 9B
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
mistral-7B-test-v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Opencsg Starcoder2 15b v0.1
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
SO
SOLAR 10.7B Instruct v1.0 Uncensored
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF

Other quants

Keep browsing