Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
CA
CarbonVillain en 10.7B v3
Carbonvillain · LiteLLMs
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
FLUX 3 Image Klein TE Uncensored
Qwen · hellmans
Params4.02B
Context256K
Smallest2.5 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
MI
MiniMax-Music
Minimax · agentory
Params25M
Context—
Smallest0.1 GB
SM
jbaron34 SmolLM2 135M Bebop Reranker
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
QW
Aether-Qwen2-0.5B-SFT-v0.0.2
Qwen · thesven
Params494M
Context128K
Q4_K_M0.4 GB
IN
interview eval zephyr 7b math train
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OR
Yi Ko 6B Orca Alpaca Gpt4 Math Lora
Orca · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
EL
Eleusis 12B
Eleusis · PygmalionAI
Params12.2B
Context1000K
Smallest4.9 GB
LL
RuterNorway Llama 2 13b chat norwegian
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
Mistral 7b v0.1 Layla v4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
ahmedheakl qwen2.5 0.5b llamafac 500k
Llama · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
DE
DeepSeek V4.1 Flash 1017 UNCENSORED MTP Maya Q4 K IQ4
DeepSeek · Adolphsson
Params485M
Context—
Smallest471.6 GB
QW
Huihui Qwen3.8 27B Abliterated
Qwen · Astark-40
Params27.3B
Context256K
Smallest10.9 GB
GP
gpt2-medium-650k-stable-diffusion-prompt-generator
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
MI
mistral small 3.1 24b instruct 2503
Mistral · mrfakename
Params23.6B
Context32K
Smallest8.9 GB
LL
georgesung llama2 7b chat uncensored
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
ST
Stablecode Completion Alpha 3b 4k
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
QW
Josiefied Qwen2.5 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
rsrishi10 gemma text2sql Instruct Finetune test
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GE
generalbakudan gemma 2b instruct ft medical qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
brunosdorneles llama3.1 8B ips realtime multilanguage
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
EV
Evillain StarDust 20B v0.2
Evillain · RichardErkhov
Params20B
Context4K
Q4_K_M4.4 GB
GR
Granite 8b Qiskit
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
GE
Gemma 2 9b It DPO
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
MI
mia llm autotrain bloom 17b sft slm
Mia · RichardErkhov
Params2.24B
Context4K
Q4_K_M1.5 GB
CO
CodeV CL 7B
Codev · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
gguf Llama 160M Chat v1
Llama · Felladrin
Params162M
Context2K
Smallest0.1 GB
PH
Phi 3 Medium 4k Instruct Abliterated v3
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB