Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b2.0
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Ottis llama3.2 arcLoRaFT
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
ReAligned Qwen3.5 2B
Qwen · Lazarus-Ai
Params1.88B
Context256K
Smallest1.0 GB
ReasoningTool use
LL
T3Q LLM Product T3Q LLM2 Solar 10.7B v1.0
Llm · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
QW
Lambent qwen2.5 reinstruct alternate lumen 14B
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
AL
AlphaMist7B Slr v2
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 8b Gemini Point21 60 Wo Cot
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Syouf llamaWithoutGuff
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 2 7b Pruned70 Retrained
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Prosparse Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CO
CompassJudger 1 32B Instruct
Compassjudger · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
GE
Google Gemma 2 27b
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
HA
haes95 cdlm 7 ko nl2sql v1.0
Haes · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
smol_llama-220M-openhermes
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
Tool use
GE
Ruqiya gemma2 2b fine tuned arabic dataset
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.7
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
DI
diffusionfamily diffullama
Diffusionfamily · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NX
Nxcode-CQ-7B-orpo
Nxcode · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
JA
japanese-stablelm-instruct-ja_vocab-beta-7b
Japanese · RichardErkhov
Params6.88B
Context4K
Q4_K_M4.2 GB
GGUF
ME
Meltemi 7B Instruct v1
Meltemi · RichardErkhov
Params7.48B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Devarui379 VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
SA
SambaLingo Arabic Chat 70B
Sambalingo · RichardErkhov
Params69.4B
Context4K
Q4_K_M41.7 GB
GGUF
EL
EleutherAI_pythia 6.9b Deduped__Sft__Tldr
Eleutherai · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
LL
torchtune_1B_full_finetuned_llama3.2_millfield_241219_meta_header_word_3epoch
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
BE
beowolx CodeNinja 1.0 OpenChat 7B
Beowolx · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing