General GGUF models

134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
EU
Euryale 1.3 longLORA 70b rope8 32k fp16 i1
Euryale · mradermacher
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
ER
ERNIE 4.5 21B A3B PT
Ernie · bartowski
Params21.8B
Context128K
Q4_K_M13.5 GB
GGUF
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CA
CarbonVillain En 10.7B v4
Carbonvillain · mradermacher
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TH
Thaki ASR KO Dialects v1 i1
Thaki · mradermacher
Params1.72B
Context64K
Q4_K_M1.1 GB
GGUF
AM
AmberChat
Amberchat · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
EK
Ekbasis Skopos 27B
Ekbasis · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
LL
Llama 2 70B OASST 1 200
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
IN
Internlm2 Math Plus 1_8b
InternLM · mradermacher
Params1.89B
Context8K
Q4_K_M1.2 GB
GGUF
LU
luxia 21.4b alignment v1.0
Luxia · MaziyarPanahi
Params21.4B
Context32K
Q4_K_M12.9 GB
GGUF
RW
rwitz_go-bruins
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
Medgemma 27b Text It Abliterated
Medgemma · mradermacher
Params27B
Context128K
Q4_K_M16.5 GB
GGUF
LL
dolphin 2.9.4 llama3.1 8b Q4 K M
Llama · yaneivan
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
KA
karakuri-lm-8x7b-instruct-v0.1
Karakuri · mmnga
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
SM
SmolLM2 360M Instruct
SmolLM · unsloth
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
NE
models--automerger--NeuralsirkrishnaInex12-7B
Neuralsirkrishna · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
LL
Llama-3-8B-Instruct-262k
Llama · crusoeai
Params8B
Context256K
Q4_K_M4.9 GB
GGUF
MI
Mistral Large Instruct 2407
Mistral · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
QW
Qwen1.5-7B-Chat-AWQ-fp16
Qwen · Qwen
Params7.72B
Context32K
Q4_K_M4.8 GB
GGUF
GE
Gemma 3 4B it Uncensored
Gemma · lemuralabs
Params3.88B
Context128K
Smallest1.2 GB
GGUF
DO
DoItYourself v1 2B
Doityourself · theprint
Params1.94B
Context256K
Q4_K_M1.3 GB
GGUF
HE
Hermes 2 DPO WestLake 7B v2
Hermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
KI
Kimi-K3
Kimi · hellohazime
Params1810B
Context1024K
Smallest441.4 GB
GGUF
CY
CyberStrike OffSec 35B
Cyberstrike · ahmedandaloes
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
GR
Granite 4.0 H Small
Granite · bartowski
Params32.2B
Context1024K
Q4_K_M19.7 GB
GGUF
LL
Nous Hermes Llama 2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
Tool use
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF

Other uses

Keep browsing