Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
SM
HuggingFaceTB_SmolLM3 3B
SmolLM · Alcoft
Params3.08B
Context64K
Q4_K_M1.9 GB
MA
Matter 0.1 7B Boost
Matter · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
TinyLlama 1.1B intermediate step 1431k 3T
Tinyllama · andrijdavid
Params1.1B
Context2K
Q4_K_M0.7 GB
PH
tuong nguyen prd Phi 3.5 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
SA
SambaLingo Arabic Base
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
BI
BioMistral BioMistral 7B TIES
Biomistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RO
roadz dv finetuned 211124
Roadz · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
liyimengwork llama 3.1 8B wenyanwen2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
LI
lightblue reranker 0.5 bincont filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
QW
Qwen3.5 0.8B
Qwen · titan0115
Params752M
Context256K
Q4_K_M0.5 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · IMUGLYHUH
Params1.86B
Context256K
Smallest0.9 GB
PY
Pygmalion 3 12b
Pygmalion · PygmalionAI
Params12.2B
Context1000K
Smallest4.9 GB
LL
DarkIdol Llama 3.1 8B Instruct 1.2 Uncensored
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
DeepSeek V4 Flash Vision Uncensored 3.87bpw Q3 K
DeepSeek · MetaphoricalCode
Params284B
Context1024K
Smallest137.6 GB
SA
saishf SOVLish Maid L3 8B
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PR
PranavHarshan NarraLarge
Pranavharshan · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
CO
Yi Coder 9B Chat Instruct TIES
Coder · RichardErkhov
Params28.3B
Context128K
Q4_K_M17.1 GB
LL
namespace Pt Llama 3 8B Instruct 80K QLoRA Merged
Llama · RichardErkhov
Params8.03B
Context80K
Q4_K_M4.9 GB
YI
Yi-1.5-9B-Chat
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
ST
Starling-LM-11B-alpha
Starling · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Configurable-Llama-3-8B-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
AN
andykcheng unitrend model 8b vllm
Andykcheng · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
deepseek-coder-1.3b-typescript
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
PE
PersianLLaMA 13B Instruct
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
CO
CodeQwen1.5 7B Trojan Clas2024 Development
Codeqwen · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
QW
Josiefied Qwen2.5 1.5B Instruct Abliterated v3
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
YA
yanolja EEVE Korean Instruct 10.8B v1.0
Yanolja · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
DE
DeepSeek V2 Lite Chat
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
MI
K2S3 Mistral 7b v1.2
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB