Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
CodeLlama 13b Instruct Hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
Bitnet-Mistral.0.2-330m-v0.2-grokfast-v2.9
Mistral · RichardErkhov
Params330M
Context4K
Q4_K_M0.2 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct Base
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
GGUF
SM
SmolLM2-135M-Instruct-K12-5000
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
IN
interview eval zephyr 7b math case 8 fixed
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral Instruct Mcqa Circuit Broken
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 3.0 3b A800M Base
Granite · RichardErkhov
Params3.37B
Context4K
Q4_K_M2.1 GB
GGUF
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b2.0
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Ottis llama3.2 arcLoRaFT
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
ReAligned Qwen3.5 2B
Qwen · Lazarus-Ai
Params1.88B
Context256K
Smallest1.0 GB
ReasoningTool use
LL
T3Q LLM Product T3Q LLM2 Solar 10.7B v1.0
Llm · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
QW
Lambent qwen2.5 reinstruct alternate lumen 14B
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
AL
AlphaMist7B Slr v2
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 8b Gemini Point21 60 Wo Cot
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Syouf llamaWithoutGuff
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 2 7b Pruned70 Retrained
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Prosparse Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CO
CompassJudger 1 32B Instruct
Compassjudger · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
GE
Google Gemma 2 27b
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
HA
haes95 cdlm 7 ko nl2sql v1.0
Haes · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
smol_llama-220M-openhermes
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
Tool use
GE
Ruqiya gemma2 2b fine tuned arabic dataset
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.7
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
DI
diffusionfamily diffullama
Diffusionfamily · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NX
Nxcode-CQ-7B-orpo
Nxcode · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF

Other quants

Keep browsing