GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
NA
NaNovel 35B A3B
Nanovel · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
QW
Qwen3-4B-Nymphaea-RP
Qwen · mradermacher
Params4B
Context256K
Best fitQ8_0 6.0 GB
Roleplay
LF
LFM2.5 2.6B UNCENSORED ABLITERATED PHILADELPHIA CLASS
Lfm · mradermacher
Params2.7B
Context125K
Best fitQ6_K 4.7 GB
ReasoningTool use
VI
VieNeu TTS v2 Turbo
Vieneu · pnnbao-ump
Params112M
Context4K
Smallest0.2 GB
Coding
MA
MagpieLM 8B Chat v0.1 i1
Magpielm · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
NE
Nemotron 3.5 Embed 1B BF16
Nemotron · W4D
Params1.14B
Context256K
Best fitF32 5.6 GB
EmbeddingVision
SW
Swallow MS 7b Instruct v0.1
Swallow · RichardErkhov
Params7.33B
Context4K
Best fitQ4_K 6.0 GB
GGUF
WI
winter garden 7b alpha
Winter · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
GGUF
CA
CantoneseLLM v2.0 30B A3B Thinking Chat Vector Merged
Cantonesellm · mradermacher
Params30.5B
Context40K
Q4_K_M18.6 GB
Reasoning
LL
Llama-3-8B-Instruct-MopeyMule
Llama · legraphista
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
MI
midashenglm 0.6b
Midashenglm · mispeech
Params596M
Context40K
Best fitBF16 2.6 GB
CodingVision
BI
Bielik 7B Instruct v0.1
Bielik · speakleash
Params7.24B
Context4K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen3 VL 4B Instruct Abliterated v1
Qwen · prithivMLmods
Params4B
Context256K
Best fitQ8_0 6.0 GB
Vision
QW
Qwable v1
Qwable · bartowski
Params35.5B
Context256K
Q4_K_M21.9 GB
Tool useVision
MI
leolm_leo-mistral-hessianai-7b-chat
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
BR
Breeze 7B 32k Instruct V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Best fitQ4_0 5.9 GB
GGUF
ST
Starcoderplus
StarCoder · mradermacher
Params15.8B
Context8K
Best fitIQ2_S 5.9 GB
Coding
MO
Morpho 1.5B Distill
Morpho · mradermacher
Params1.54B
Context128K
Best fitQ6_K 2.0 GB
GGUF
OV
OvisOCR2
Ovisocr · bartowski
Params752M
Context256K
Best fitBF16 2.4 GB
Vision
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
DE
DeepSeek R1 Distill Qwen 1.5B uncensored
DeepSeek · mradermacher
Params1.78B
Context128K
Best fitF16 4.3 GB
Reasoning
QW
Qwen3 1.7b Abliterated
Qwen · mradermacher
Params1.72B
Context40K
Best fitQ6_K 2.9 GB
GGUF
LF
LFM2.5 24B A2B Uncensored
Lfm · mradermacher
Params24B
Context128K
Q4_K_M14.5 GB
ReasoningTool use
MI
mistral 7b orpo capybara reproduction
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
GGUF
HY
Hy-Mt2-1.8B
unsloth
Params1.79B
Context256K
Best fitBF16 4.6 GB
GGUF
QW
Qwen3-Vl-8B-Instruct-1M
Qwen · unsloth
Params8.19B
Context977K
Best fitQ3_K_XL 6.0 GB
Vision
LO
Loopl 0.8b
Loopl · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
Tool useVision
LF
LFM2 VL 3B
Lfm · LiquidAI
Params2.57B
Context125K
Best fitQ8_0 5.2 GB
Vision
MI
cognitivecomputations_dolphin-2.6-mistral-7b-dpo
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
CodingTool use
MI
RenCoder Ministral 8B Instruct 2410
Mistral · mradermacher
Params8B
Context32K
Best fitQ3_K_L 6.0 GB
Coding

Other memory sizes

Keep browsing