Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Flammen
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
ticlazau granite 8b instruct rpgle
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
LL
llama 3 chinese 8b instruct
Llama · hfl
Params8.03B
Context8K
Smallest3.2 GB
LL
ELYZA-japanese-CodeLlama-7b-instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
NO
notHumpback-M1
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
CN
cnmoro TinyLlama ContextQuestionPair Classifier Reranker
Cnmoro · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
HuggingFaceH4 mistral 7b anthropic
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
ahmedheakl asm2asm deepseek 1.3b 500k x86 O0 arm gnueabi gcc
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GI
Gille StrangeMerges 51 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AL
AlphaMist7B Slr v1
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
FunDialogues llamav2 LoRaco 7b merged
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
mlfoundations dev hp ablations mistral bsz2048 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
LL
Llama-3-8B-Lexi-Uncensored
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Huihui Qwen3.6 35B A3B Claude 4.7 Opus abliterated
Qwen · Momix-44
Params34.7B
Context256K
Smallest13.6 GB
SW
Swallow MX 8x7b NVE v0.1
Swallow · RichardErkhov
Params46.7B
Context4K
Q4_K_M28.4 GB
EP
epiCRealism SD15
Epicrealism · ygohel18
Params1.07B
Context—
Smallest1.6 GB
SW
Swallow 7b Hf
Swallow · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
VI
vicuna 13b v1.5 16k
Vicuna · shaowenchen
Params13B
Context16K
Smallest5.4 GB
OL
OLMoE 1B 7B 0924 Instruct
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
ST
Starcoder2 15b Fp16
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
PY
pythia_2.8b_deduped-task065_timetravel_consistent_sentence_classification
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
PH
microsoft Phi 3 medium 128k instruct
Phi · RichardErkhov
Params14B
Context128K
Q4_K_M8.6 GB
LL
jc9080 llama3 open ko 8b Instruct shimshimi 500 ver2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama PLLuM 70B Chat
Llama · Nondzu
Params70.6B
Context128K
Q4_K_M42.5 GB
BI
BioMistral BioMistral 7B DARE
Biomistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
LL
Llama3.2 1b Train Aware Quantization
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
NO
nota ai st vicuna v1.3 5.5b ppl
Nota · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
TE
Tensoic TinyLlama 1.1B 2.5T openhermes
Tensoic · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB