Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
MiniLLM Gpt2 120M
Minillm · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
inswave AISquare Instruct llama2 koen 13b v0.9.5
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
MI
AlexeyRyzhikov mistral7b text to json v5.20
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
PH
fractalego wafl phi3 mini 4k
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
MI
mistral-7b-ft-h4-no_robots_instructions
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GL
Glm 4 9b
GLM · RichardErkhov
Params9.4B
Context8K
Q4_K_M6.3 GB
TA
Taiwan LLM 13B v2.0 Chat
Taiwan · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
TO
toastloaf autotrain gpt2 finetune crab
Toastloaf · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
SM
SmolLM2-135M-FT-SCP-Wiki
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange proofassistants
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
IM
P H3 Image
Image · AtomicChat
Params20.2B
Context—
Smallest9.0 GB
GE
gemma-2b-translation-v0.150
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
Qwen2 7B Instruct Refine
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
AI
aisha partha medical question and answer gpt2
Aisha · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
Magnum Instruct DPO
Magnum · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GE
Gemma-Wukong-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
terry69 qwen feedback dirty 0.5
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
MI
Mistral Nemo Instruct 2407
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
Llama 3.2 Korean Bllossom 3B
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
DI
distilgpt2-tiny-conversational
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
DE
Deepseek Coder 7b Base v1.5
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BI
bitnet_b1_58-3B
Bitnet · RichardErkhov
Params3.32B
Context2K
Q4_K_M2.5 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB