General GGUF models
134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.
JA
AI21 Jamba2 Mini Ultra Uncensored Heretic
Jamba · llmfan46
Params51.6B
Context256K
Q4_K_M31.2 GB
YI
Nous Hermes 2 Yi 34B
Yi · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
GE
Gemma 3 270m It
Gemma · bartowski
Params268M
Context32K
Q4_K_M0.3 GB
NO
Noromaid 20B v0.1.1
Noromaid · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
AH
Ahma 7B
Ahma · mradermacher
Params7B
Context2K
Q4_K_M4.3 GB
MI
Infinity Instruct 7M 0729 Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
SO
SOLAR 10.7B Instruct v1.0
Solar · TheBloke
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen3.6 35B A3B REAP 48 v2
Qwen · crucible-labs
Params19.2B
Context256K
Smallest9.4 GB
IN
Index Nailong 9B
Index · mradermacher
Params8.95B
Context224K
Q4_K_M5.6 GB
MA
mamba 130m hf
Mamba · QuantFactory
Params129M
Context1024K
Q4_K_M0.1 GB
LF
LFM2.5 24B A2B Nyquant Q4 preview
Lfm · XGeorgeCostanzaX
Params24B
Context128K
Smallest12.0 GB
VI
T VisStar 7B v0.1 i1
Visstar · mradermacher
Params7.29B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.8 27B Escha W2
Qwen · aj9o9
Params6.33B
Context256K
Smallest2.9 GB
MI
Mistral-11B-SynthIAirOmniMix
Mistral · mradermacher
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Hermes 2 Theta Llama 3 8B
Llama · NousResearch
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 7B Instruct abliterated v2
Qwen · kepom
Params7.6B
Context32K
Q4_K_M4.7 GB
PH
phi3.5 gutenberg 4B i1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.4 GB
SA
vagosolutions_sauerkrautlm-7b-hero
Sauerkrautlm · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
CY
TheDrummer Cydonia 24B v2
Cydonia · bartowski
Params23.6B
Context32K
Q4_K_M14.3 GB
AI
airoboros-65b-gpt4-m2.0
Airoboros · mradermacher
Params65B
Context2K
Q4_K_M39.3 GB
LL
Llama3-ChatQA-1.5-8B
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
TM
Tmax v1.1 9b
Tmax · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
TR
Trurl 2 13B
Trurl · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
SW
Swallow-70b-NVE-instruct-hf
Swallow · mradermacher
Params69B
Context4K
Q4_K_M41.4 GB
ST
stablelm tuned alpha 7b i1
Stablelm · mradermacher
Params7.87B
Context4K
Q4_K_M5.1 GB
GU
Guanaco 13B Uncensored i1
Guanaco · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
QW
Qwen3 14B
Qwen · lm-kit
Params14.8B
Context40K
Q4_K_M9.0 GB
DE
Decision 2.0 Vega 27B
Decision · vllm-sr
Params25.6B
Context32K
Q4_K_M15.9 GB
FI
Fimbulvetr 11B v2 i1
Fimbulvetr · mradermacher
Params10.7B
Context4K
Q4_K_M6.6 GB
QW
Qwen3 1.7B
Qwen · gaianet
Params2.03B
Context40K
Q4_K_M1.3 GB