Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
kimleang123 qwen 7b v2 khmer qa 1000 v10 merge model attn flash 2
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
KO
JY623 KoSOLAR 10.7B merge v3.0
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
ruslandev llama 3 8b gpt 4o ru1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
SFR Iterative DPO LLaMA 3 8B R
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GL
Glm 4 9b Chat
GLM · RichardErkhov
Params9.4B
Context128K
Q4_K_M6.3 GB
Tool use
LL
yujiepan llama 3 tiny random
Llama · RichardErkhov
Params14M
Context—
Q4_K_M0.0 GB
GGUF
SM
akhilfau fine tuned smolLM2 physics 10
SmolLM · RichardErkhov
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
IN
internlm2-limarp-chat-20b
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
GE
Gemma 2 2b Jpn It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
GP
gpt2-small-italian-embeddings
Gpt · RichardErkhov
Params132M
Context1K
Q4_K_M0.1 GB
Embedding
CP
cpt_st-vicuna-v1.3-1.5b-ppl
Cpt · RichardErkhov
Params1.48B
Context2K
Q4_K_M0.9 GB
GGUF
MI
Nondzu Mistral 7B code 16k qlora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
tyson0420 stack codellama 7b inst
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
KH
KhanhTTS OmniVoice
Khanhtts · doof-ferb
Params—
Context—
Smallest1.0 GB
GGUF
NO
nota ai st vicuna v1.3 10.5b taylor
Nota · RichardErkhov
Params10.5B
Context2K
Q4_K_M6.3 GB
GGUF
QW
Qwen3.5 0.8B Base
Qwen · titan0115
Params752M
Context256K
Q4_K_M0.5 GB
ReasoningTool use
SH
Shwetaupadhyay7878 gita text generation gpt2
Shwetaupadhyay · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Open_Hermes_Orca_Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
HA
harry_potter_chatbot
Harry · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
CS
csg-wukong-1B-chat-v0.1
Csg · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
PH
disi unibo nlp phi3.5 SFT medqa triples nographeval cot
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
PH
IntelLabs sqft sparsepeft phi 3 mini 4k 50 math heu
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
meta llama CodeLlama 7b Instruct hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NF
nfaheem Marcoroni 7b DPO Merge
Nfaheem · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IS
ishantd unyielding mustard agouti
Ishantd · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ME
megatron-gpt2-345m-evol_instruct_v2
Megatron · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
simonycl llama 3.1 8b instruct ultrafeedback agg judge
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Other quants

Keep browsing