Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
PatronusAI Llama 3 Patronus Lynx 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
FU
FuseChat 7B VaRM
Fusechat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MR
MrRobotoAI Hathor v3.3
Mrrobotoai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
OpenELM-1_1B
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
QW
mega asr
Qwen · cstr
Params2.35B
Context—
Smallest0.0 GB
GGUF
HO
K2 Horizon MoVA 36B A4B
Horizon · eaddario
Params37.4B
Context512K
Smallest0.3 GB
GGUF
LO
longchat 13b 16k
Longchat · shaowenchen
Params13B
Context16K
Smallest5.4 GB
GGUF
YI
Yi-1.5-9B
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
LongWriter Llama3.1 8b
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
RA
RakutenAI-7B-chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
LZ
lzw1008 Emollama 7b
Lzw · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
MO
MobileLLaMA 1.4B Base
Mobilellama · andrijdavid
Params1.36B
Context2K
Q4_K_M0.8 GB
GGUF
NE
NeuralMona_MoE-4x7B
Neuralmona · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
GGUF
WA
Wan2.2 Animate 14B
Wan · befox
Params17.3B
Context—
Smallest6.8 GB
GGUF
WA
Wanfq FuseLLM 7B
Wanfq · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
AU
AutoCoder_S_6.7B
Autocoder · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
FL
flammenai flammen15 gutenberg DPO v1 7B
Flammenai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
RESMPDEV Qwen2 Wukong 5B Instruct Sliced
Qwen · RichardErkhov
Params5.75B
Context32K
Q4_K_M3.6 GB
GGUF
AG
AgentEvol 7B
Agentevol · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
OL
OLMo 1B 0724 Hf
OLMo · aifoundry-org
Params1.28B
Context4K
Smallest0.3 GB
GGUF
PO
Poro 34B Chat
Poro · RichardErkhov
Params35.1B
Context7K
Q4_K_M22.4 GB
GGUF
GE
thusinh1969 gemma2 2b rerank checkpoint 8800
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
Embedding
LL
scb10x llama 3 typhoon v1.5x 8b instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
TheDrummer Big Tiger Gemma 27B v1
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
LL
Laim Llama 3.1 MedPalm2 imitate 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
AL
alpindale pygmalion instruct
Alpindale · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
CH
CheckGPT SOLAR 10.7B
Checkgpt · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
NE
IQ2 NL IQ3 NL Test Models
Nemotron · eaddario
Params8.89B
Context1024K
Smallest0.0 GB
GGUF
MO
MOSS Transcribe preview 2B
Moss · cstr
Params2.42B
Context—
Smallest0.0 GB
GGUF
LL
Llama 3 6B v0
Llama · RichardErkhov
Params6.29B
Context8K
Q4_K_M3.9 GB
GGUF

Other quants

Keep browsing