Q6_K GGUF models
6-bit k-quant. Very close to the original model for most uses. 112449 models have a Q6_K file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Huihui Qwen3 30B A3B Instruct 2507 Abliterated
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
SE
SeedVR2
Seedvr · cmeka
Params3.39B
Context—
Q4_K_M2.0 GB
MI
Models Mistralai Ministral 3 3B Reasoning 2512
Mistral · MaziyarPanahi
Params3.43B
Context256K
Q4_K_M2.1 GB
GE
Models Google Gemma 2 2b It
Gemma · MaziyarPanahi
Params2.61B
Context8K
Q4_K_M1.7 GB
CL
Clef Flash
Clef · bartowski
Params9.1B
Context256K
Q4_K_M6.0 GB
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
MI
MiniCPM V 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Q4_K_M5.0 GB
WI
models--microsoft--WizardLM-2-7B
Wizardlm · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Models Google Gemma 3 1b It
Gemma · MaziyarPanahi
Params1B
Context32K
Q4_K_M0.8 GB
CL
Cloudflare clef
Clef · bartowski
Params27B
Context256K
Q4_K_M17.7 GB
BE
BeaverAI NeMoist 21B v1a
Beaverai · RichardErkhov
Params21B
Context—
Q4_K_M0.0 GB
QW
Qwen Image 2.1 Uncensored
Qwen · 0xSojalSec
Params7.12B
Context—
Q4_K_M4.6 GB
GR
Granite Speech 4.1 2B
Granite · handy-computer
Params2.31B
Context—
Q4_K_M1.6 GB
QW
Qwen3.8 27B Heretic JP Roleplay NSFW DanbooruTags
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
MI
mistralai_Devstral Small 2 24B Instruct 2512
Mistral · lmstudio-community
Params23.6B
Context384K
Q4_K_M14.3 GB
MI
Models Mrfakename Mistral Small 3.1 24b Instruct 2503 Hf
Mistral · MaziyarPanahi
Params23.6B
Context32K
Q4_K_M14.3 GB
FU
Fun ASR MLT Nano 2512
Fun · handy-computer
Params830M
Context—
Q4_K_M0.6 GB
LT
LTX 2
Ltx · unsloth
Params18.9B
Context—
Q4_K_M12.8 GB
LL
Hermes 3 Llama 3.1 8B
Llama · NousResearch
Params8B
Context128K
Q4_K_M4.9 GB
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · unsloth
Params14.8B
Context128K
Q4_K_M9.0 GB
YI
models--01-ai--Yi-1.5-6B-Chat
Yi · MaziyarPanahi
Params6.06B
Context4K
Q4_K_M3.7 GB
GE
Models Google Gemma 3 12b It
Gemma · MaziyarPanahi
Params11.8B
Context128K
Q4_K_M7.3 GB
LL
models--MaziyarPanahi--Llama-3-8B-Instruct-64k
Llama · MaziyarPanahi
Params8B
Context63K
Q4_K_M4.9 GB
MI
Models Mistralai Mistral Nemo Instruct 2407
Mistral · MaziyarPanahi
Params12.2B
Context1000K
Q4_K_M7.5 GB
LL
Models Meta Llama Llama 3.3 70B Instruct
Llama · MaziyarPanahi
Params70.6B
Context128K
Q4_K_M42.5 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · thesven
Params7B
Context32K
Q4_K_M4.4 GB
MI
models--mistralai--mathstral-7B-v0.1
Mistral · MaziyarPanahi
Params7.25B
Context32K
Q4_K_M4.4 GB
WH
Whisper Large
Whisper · handy-computer
Params1.54B
Context—
Q4_K_M1.0 GB
MI
Rocinante X 12B v1
Mistral · TheDrummer
Params12.2B
Context128K
Q4_K_M7.5 GB