Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
MiniMax Music3
Minimax · joemattie
Params25M
Context—
Smallest0.1 GB
GGUF
MI
Nondzu Mistral 7B code 16k qlora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
tyson0420 stack codellama 7b inst
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
KH
KhanhTTS OmniVoice
Khanhtts · doof-ferb
Params—
Context—
Smallest1.0 GB
GGUF
NO
nota ai st vicuna v1.3 10.5b taylor
Nota · RichardErkhov
Params10.5B
Context2K
Q4_K_M6.3 GB
GGUF
QW
Qwen3.5 0.8B Base
Qwen · titan0115
Params752M
Context256K
Q4_K_M0.5 GB
ReasoningTool use
SH
Shwetaupadhyay7878 gita text generation gpt2
Shwetaupadhyay · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Open_Hermes_Orca_Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
HA
harry_potter_chatbot
Harry · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
CS
csg-wukong-1B-chat-v0.1
Csg · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
AL
all MiniLM L6 v2
All · cstr
Params23M
Context512
Smallest0.0 GB
Embedding
PH
disi unibo nlp phi3.5 SFT medqa triples nographeval cot
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
PH
IntelLabs sqft sparsepeft phi 3 mini 4k 50 math heu
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
meta llama CodeLlama 7b Instruct hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NF
nfaheem Marcoroni 7b DPO Merge
Nfaheem · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IS
ishantd unyielding mustard agouti
Ishantd · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ME
megatron-gpt2-345m-evol_instruct_v2
Megatron · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
simonycl llama 3.1 8b instruct ultrafeedback agg judge
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
ML
mlfoundations dev oh v1.3 opengpt x2
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
YI
Yi 6b Dpo v0.2
Yi · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
QW
Qwen3.8-27B
Qwen · Arslan2703
Params27.3B
Context256K
Smallest8.4 GB
ReasoningTool useVision
LL
Security Llama3.2 3B
Llama · prithivMLmods
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
MI
nbeerbower mistral nemo bophades 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
LL
kenhktsui llama3.2 3b instruct thinking sft merged
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
ReasoningTool use
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama-2-7b-german-assistant-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
KU
Kukedlc Fasciculus Arcuatus 7B slerp
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama-3-8B-Instruct-function-calling
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
QW
Qwen3.6-27B-Uncensored-HauhauCS-Balanced
Qwen · CCSSNE
Params26.9B
Context256K
Smallest10.0 GB
ReasoningTool useVision

Other quants

Keep browsing