Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
TO
tongliuphysics zephyr 7b ultra p 0.07
Tongliuphysics · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TU
Tulu v2.5 Dpo 13b Uf Mean
Tulu · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
FR
Fredithefish Guanaco 3B Uncensored
Fredithefish · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
SU
suehyunpark potpourri 8b inst fft induction bc trajectory max10 per task warm start
Suehyunpark · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TI
TinyLlama-1.1B-Chat-v0.4
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
chinese llama 2 7b
Llama · shaowenchen
Params6.93B
Context4K
Smallest2.9 GB
MI
osanseviero mistral instruct frankenmerge
Mistral · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
LZ
lzw1008 Emollama chat 7b
Lzw · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
PY
Pythia 1.4b
Pythia · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
RE
reflect_llm8B_om2-mstlrg300k460kLlm33130k_SftDpo-MgSpsdpIter2T1_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
EL
Elfrino HarmonicHarlequin v5 20B
Elfrino · RichardErkhov
Params33.3B
Context4K
Q4_K_M20.1 GB
DO
DongfuJiang prm math only hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
Phi 3 4B 4k Instruct CMC
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
LL
Llama 3.2 1B Instruct Sw Be Linear
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
PY
Pythia 1.4b
Pythia · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
PL
Plaban81 Moe 4x7b math reason code
Plaban · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
OR
Yi Ko 6B Orca Alpaca Gpt4 Math
Orca · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
MI
Mistral 7B Base SFT IPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
MathHermes-2.5-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
XL
xLAM 1b Fc R
Xlam · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
LL
chargoddard prometheus 2 llama 3 8b
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
microsoft rho math 1b v0.1
Microsoft · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
RE
ReflectionCoder DS 6.7B
Reflectioncoder · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
CA
CarbonVillain En 10.7B v5
Carbonvillain · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
SM
toastloaf smollm2 135m it orca agentinstruct creative
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
MA
Marsouuu lareneg3B ECE PRYMMAL Martial
Marsouuu · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
PH
IntelLabs sqft sparsepeft phi 3 mini 4k 50 cs heu
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
QW
Qwen1.5-7B-Chat
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.8 GB
MI
PiVoT 10.7B Mistral v0.2 RP
Mistral · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
MI
abhishek autotrain xva0j mixtral8x7b
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB