Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
OP
OpenELM-1_1B-Instruct
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Qwen2-57B-A14B-Instruct
Qwen · legraphista
Params57.4B
Context32K
Smallest21.1 GB
GGUF
SA
Salt Asr_Wav Uni_1_Tts_Wav Uni_1 12k
Salt · RichardErkhov
Params3.23B
Context128K
Q4_K_M2.0 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · ahmed-amedo
Params1.86B
Context—
Smallest0.9 GB
GGUF
GR
Granite 8b Code Instruct 128k
Granite · RichardErkhov
Params8.05B
Context125K
Q4_K_M4.9 GB
Coding
JI
Jina Reranker v2 Base Multilingual
Jina · gpustack
Params278M
Context1K
Q4_K_M0.2 GB
CodingEmbedding
MI
mistralai Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama3 8b Math Sft
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
DE
DeepSeek-Coder-V2-Lite-Base
DeepSeek · legraphista
Params15.7B
Context160K
Smallest5.0 GB
Coding
LL
Meta Llama 3.1 70B Instruct
Llama · legraphista
Params70.6B
Context128K
Smallest15.3 GB
GGUF
QW
Qwen2.5 14B Instruct
Qwen · legraphista
Params14.8B
Context32K
Smallest3.6 GB
GGUF
OC
octo net
Octo · NexaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
BO
Bookworm 10.7B v0.4 DPO
Bookworm · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
MI
flammenai flammen20 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
RI
RigoChat 7b v2
Rigochat · IIC
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
OP
OpenELM-450M-Instruct
Openelm · RichardErkhov
Params457M
Context2K
Q4_K_M0.3 GB
GGUF
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · li1272
Params8.2B
Context256K
Smallest3.3 GB
CodingVision
LL
Llama 3.2 1B Instruct Abliterated
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GGUF
MI
Mistral 7B ReMax v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QV
QVikhr 3 8B Instruction
Qvikhr · Vikhrmodels
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
LL
kykim0 Llama 2 7b ultrachat code
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
Coding
QW
Qwen2.5 1.5B Instruct
Qwen · legraphista
Params1.54B
Context32K
Smallest0.4 GB
GGUF
MI
Mistral Nemo 12B Instruct 2407
Mistral · starble-dev
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Llama 2 7b Ultrachat
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Higgs-Llama-3-70B
Llama · legraphista
Params70.6B
Context8K
Smallest15.3 GB
GGUF
LL
picAIso TARS 8B llama REMIX v2
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QU
QuantumIntelligence QI neural chat 7B ko DPO
Quantumintelligence · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Tifa 7B Qwen2 v0.1
Qwen · Tifa-RP
Params7.62B
Context32K
Q4_K_M4.7 GB
Roleplay
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing