Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama2_Biomed_Meditron
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
LO
Local Novel LLM project Ninja v1 NSFW 128k
Local · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FI
FireRed Image Edit 1.0 comfy
Firered · Novice25
Params20.4B
Context—
Smallest7.5 GB
GGUF
LL
AIML GEEK mini prompt finetuned CodeLlama7b
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
CO
Yi Coder 1.5B Chat Jlama
Coder · RichardErkhov
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
VI
vitumafeni tiny crypto sentiment analysis
Vitumafeni · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · saracen9
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
LL
Llama 3.2 3b It ComfyUIPrompt
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
SA
saishf SOVL Mega Mash V2 L3 8B
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
kykim0 Llama 2 7b ultrachat200k 2e
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M2.0 GB
GGUF
CO
Codegemma 2b
Codegemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
speechless-mistral-7b-dare-0.85
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DI
Distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
Coding
TI
TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
abhishek autotrain mixtral7x8b math
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
LL
Llama 3 10b Ko 240604 E2F
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
GGUF
ML
mlfoundations dev oh dcft v1.2 no curation gpt 4o mini wo metamath
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
CodeLlama-13B-Instruct-fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
HW
hwkwon S SOLAR 10.7B SFT v1.3
Hwkwon · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
DO
dolphin-ultrafeedback-dpo
Dolphin · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LG
lgaalves gpt2 camel physics platypus
Lgaalves · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
DE
ahmedheakl asm2asm deepseek 1.3b 100k x86 arm gcc
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
FL
FLUX.1 Canny dev
Flux · second-state
Params11.9B
Context—
Smallest4.2 GB
GGUF
FR
Fredithefish Guanaco 3B Uncensored v2
Fredithefish · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
ID
ideogram 4
Ideogram · Abiray
Params9.28B
Context—
Smallest5.6 GB
GGUF
FA
Falcon
Falcon · RichardErkhov
Params5.47B
Context2K
Q4_K_M3.4 GB
GGUF
MI
Mistral 7B v0.2 Ko Y24_v2.0
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing