Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
DE
deepseek-coder-1.3b-chat
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
MI
mii community zefiro 7b base ITA
Mii · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LF
lfm2 audio 1.5b jp
Lfm · cstr
Params79M
Context—
Smallest0.2 GB
GGUF
DE
DeepSeek Coder V2 Lite Base
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
MI
Flammen
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
stt uz fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
GGUF
GR
ticlazau granite 8b instruct rpgle
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
GGUF
LL
llama 3 chinese 8b instruct
Llama · hfl
Params8.03B
Context8K
Smallest3.2 GB
GGUF
LL
ELYZA-japanese-CodeLlama-7b-instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NO
notHumpback-M1
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
CN
cnmoro TinyLlama ContextQuestionPair Classifier Reranker
Cnmoro · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Embedding
MI
HuggingFaceH4 mistral 7b anthropic
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HO
Hojo-ASR-Multi-V1
Hojo · cstr
Params4.81B
Context—
Smallest4.4 GB
GGUF
DE
ahmedheakl asm2asm deepseek 1.3b 500k x86 O0 arm gnueabi gcc
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
GI
Gille StrangeMerges 51 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AL
AlphaMist7B Slr v1
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
FunDialogues llamav2 LoRaco 7b merged
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
mlfoundations dev hp ablations mistral bsz2048 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use
LL
Llama-3-8B-Lexi-Uncensored
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SW
Swallow MX 8x7b NVE v0.1
Swallow · RichardErkhov
Params46.7B
Context4K
Q4_K_M28.4 GB
GGUF
SW
Swallow 7b Hf
Swallow · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
VI
vicuna 13b v1.5 16k
Vicuna · shaowenchen
Params13B
Context16K
Smallest5.4 GB
GGUF
OL
OLMoE 1B 7B 0924 Instruct
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
GGUF
ST
Starcoder2 15b Fp16
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
GL
lid-fasttext-glotlid-v3
Glotlid · cstr
Params419M
Context—
Smallest0.2 GB
GGUF
PY
pythia_2.8b_deduped-task065_timetravel_consistent_sentence_classification
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
PH
microsoft Phi 3 medium 128k instruct
Phi · RichardErkhov
Params14B
Context128K
Q4_K_M8.6 GB
GGUF
LL
jc9080 llama3 open ko 8b Instruct shimshimi 500 ver2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama PLLuM 70B Chat
Llama · Nondzu
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF

Other quants

Keep browsing