Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
RO
RoLlama2-7b-Chat
Rollama · legraphista
Params6.74B
Context4K
Smallest1.5 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Mrt45
Params34.7B
Context256K
Q4_K_M21.2 GB
LL
Llama 3 8B Instruct 262k
Llama · Monor
Params8.03B
Context256K
Q4_K_M4.9 GB
DE
DeepCoder 14B Preview
Deepcoder · Joumdane
Params14.8B
Context128K
Q4_K_M9.0 GB
NO
nothingiisreal MN 12B Starcannon v3
Nothingiisreal · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
scb10x llama 3 typhoon v1.5x 70b instruct
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
LLaMA 3 8B SFR Iterative DPO R
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
VI
vicgalle ConfigurableHermes 7B
Vicgalle · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GT
Gte Multilingual Reranker Base
Gte · gpustack
Params306M
Context8K
Q4_K_M0.2 GB
PH
Phi 3.5 Mini Instruct
Phi · legraphista
Params3.82B
Context128K
Smallest0.8 GB
LL
Llama 3 Instruct 8B SimPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
DE
Husky v1 Math Deepseekmath 7b Instruct
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
LL
chinese llama 2 7b
Llama · hfl
Params6.93B
Context4K
Smallest2.6 GB
MI
nbeerbower bophades mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RA
RakutenAI 7B chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CH
ChaoticNeutrals Prima LelantaclesV5 7b
Chaoticneutrals · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen_Qwen3 4B Instruct 2507
Qwen · Alcoft
Params4B
Context256K
Q4_K_M2.4 GB
LL
Llama 3 Swallow 70B v0.1
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
SP
SparseLlama 3 8B Pruned_50.2Of4
Sparsellama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
PO
Poro 34B
Poro · RichardErkhov
Params34B
Context7K
Q4_K_M22.4 GB
QW
Qwen2 VL 2B Instruct
Qwen · turingevo
Params1.54B
Context32K
Q4_K_M1.0 GB
SM
SmallThinker 21BA3B Instruct
Smallthinker · Tiiny
Params21.5B
Context16K
Q4_K_M13.1 GB
SM
gguf SmolLM 135M Instruct
SmolLM · Felladrin
Params135M
Context2K
Smallest0.1 GB
ME
MetaMath Bagel DPO 34B
Metamath · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
OL
OLMo 7B Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
VI
vinallama-2.7b
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
CY
z image cyberrealistic v8
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
MI
Ministral 3 14B Instruct 2512
Mistral · eaddario
Params13.5B
Context256K
Q4_K_M8.2 GB