Q8_0 GGUF models

8-bit quant. Near-lossless, at roughly twice the size of a 4-bit quant. 111847 models have a Q8_0 file. Sizes run from 1M to 2780B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 VL 7B Instruct Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Huihui Qwen3 30B A3B Instruct 2507 Abliterated
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
GGUF
SE
SeedVR2
Seedvr · cmeka
Params3.39B
Context—
Q4_K_M2.0 GB
GGUF
GE
Models Google Gemma 2 2b It
Gemma · MaziyarPanahi
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
CL
Clef Flash
Clef · bartowski
Params9.1B
Context256K
Q4_K_M6.0 GB
CodingVision
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
MI
MiniCPM V 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Q4_K_M5.0 GB
CodingVision
GE
Gemma 4 E4B It OBLITERATED v3.1
Gemma · OBLITERATUS
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
WI
models--microsoft--WizardLM-2-7B
Wizardlm · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Models Google Gemma 3 1b It
Gemma · MaziyarPanahi
Params1B
Context32K
Q4_K_M0.8 GB
GGUF
CL
Cloudflare clef
Clef · bartowski
Params27B
Context256K
Q4_K_M17.7 GB
CodingVision
VI
VibeVoice-7B
Vibevoice · audio-cpp
Params7B
Context—
Smallest10.6 GB
GGUF
BE
BeaverAI NeMoist 21B v1a
Beaverai · RichardErkhov
Params21B
Context—
Q4_K_M0.0 GB
GGUF
QW
Qwen Image 2.1 Uncensored
Qwen · 0xSojalSec
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
GR
Granite Speech 4.1 2B
Granite · handy-computer
Params2.31B
Context—
Q4_K_M1.6 GB
GGUF
MI
mistralai_Devstral Small 2 24B Instruct 2512
Mistral · lmstudio-community
Params23.6B
Context384K
Q4_K_M14.3 GB
CodingVision
MI
Models Mrfakename Mistral Small 3.1 24b Instruct 2503 Hf
Mistral · MaziyarPanahi
Params23.6B
Context32K
Q4_K_M14.3 GB
GGUF
FU
Fun ASR MLT Nano 2512
Fun · handy-computer
Params830M
Context—
Q4_K_M0.6 GB
GGUF
QW
Qwen3 TTS 12Hz 1.7B Base
Qwen · ggml-org
Params1.73B
Context32K
Q4_K_M1.0 GB
Vision
LT
LTX 2
Ltx · unsloth
Params18.9B
Context—
Q4_K_M12.8 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · NousResearch
Params8B
Context128K
Q4_K_M4.9 GB
RoleplayTool use
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · unsloth
Params14.8B
Context128K
Q4_K_M9.0 GB
Reasoning
GE
Models Google Gemma 3 12b It
Gemma · MaziyarPanahi
Params11.8B
Context128K
Q4_K_M7.3 GB
GGUF
LL
models--MaziyarPanahi--Llama-3-8B-Instruct-64k
Llama · MaziyarPanahi
Params8B
Context63K
Q4_K_M4.9 GB
GGUF
MI
Models Mistralai Mistral Nemo Instruct 2407
Mistral · MaziyarPanahi
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Models Meta Llama Llama 3.3 70B Instruct
Llama · MaziyarPanahi
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
MI
Mistral-7B-Instruct-v0.3
Mistral · thesven
Params7B
Context32K
Q4_K_M4.4 GB
GGUF
MI
models--mistralai--mathstral-7B-v0.1
Mistral · MaziyarPanahi
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
WH
Whisper Large
Whisper · handy-computer
Params1.54B
Context—
Q4_K_M1.0 GB
GGUF
MI
Rocinante X 12B v1
Mistral · TheDrummer
Params12.2B
Context128K
Q4_K_M7.5 GB
GGUF

Other quants

Keep browsing