Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Lllma 3.2 1B
Lllma · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LO
LordNoah Alpaca refine gpt2 e1 se0
Lordnoah · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
PH
parsak phi 2 code instruct
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
LL
Nov2_Oasst_Aft_Llama_Lr_3E 5
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
AE
aetherwiing MN 12B Starsong v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
MI
gqd mistral merge 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
nbeerbower Lyra Gutenberg mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
YI
dolphin-2.9.1-yi-1.5-9b
Yi · RichardErkhov
Params8.83B
Context12K
Q4_K_M5.3 GB
GE
SangMoone merge gemma2 2b it difficulty hard
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
jan hq Llama3.1 s instruct 2024 09 10 v2 cp 1000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
EU
Eurus 7b Kto
Eurus · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 7b Chat Hf Gpt 3.5 80k Base_Lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
CU
curiousily tiny crypto sentiment analysis
Curiousily · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
kajuyerim Llama 3.1 8B Instruct Financial Sentiment Analysis
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GI
Gille StrangeMerges 53 7B model stock
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MY
mychen76 openmixtral 6x7b v2
Mychen · RichardErkhov
Params35.4B
Context32K
Q4_K_M21.4 GB
LL
CodeLlama 7B Instruct Fp16
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
dalau627 text to sql finetuned mistral 7b
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
HE
Hello Bible Gabriel 8x7B Instruct v0.1
Hello · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
PH
IntelLabs sqft phi 3 mini 4k 40 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
SA
SashaSheykina gpt2 finetuned cXg nl to code
Sashasheykina · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
VO
VoxCPM2
Voxcpm · NANI-Nithin
Params—
Context—
Smallest1.6 GB
LL
nnethercott llava v1.5 7b hf vicuna
LLaVA · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GE
nbeerbower gemma2 gutenberg 27B
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
LL
Llama 3 OffsetBias 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
Phi 3 Mini 4k Grammar Correction
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
ML
mlfoundations dev oh v1 2 only evol instruct
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB