Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
PH
e palmisano Phi3 ITA mini 4K instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
DE
deepseek-coder-6.7b-instruct
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LA
lamm mit PRefLexOR ORPO DPO EXO REFLECT 10222024
Lamm · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
HE
Heejindo rationale model e3 save5000 f3
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
LL
ihughes15234 llama 3 1 8bi tictactoe dpo3epoch v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
SO
SOLAR-10.7B-Instructlora_sftt-v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen 1_8B Stable Diffusion Prompt
Qwen · RichardErkhov
Params1.84B
Context8K
Q4_K_M1.2 GB
SH
Sheared-Pythia-160m-Platypus
Sheared · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
QW
DFlash
Qwen · zeeksa
Params26.9B
Context256K
Q4_K_M1.0 GB
LL
amanchahar Llama Instruct Finance pre
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
PI
pip-library-etl-1.3b
Pip · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · muralcode
Params34.7B
Context256K
Q4_K_M21.2 GB
MI
MiniMax H3
Minimax · JRay0421
Params20.1B
Context—
Q4_K_M18.2 GB
TI
TinyLlama-1.1B-Telugu-Romanization-v0-Base
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
ME
MetaMath Neural Chat 7b v3 2 Ties
Metamath · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
FA
fanchul aris llm1
Fanchul · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Locutusque Hyperion 1.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Kukedlc Qwen2 1.5B Spanish 1.0
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
ME
merlinite-7b
Merlinite · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
suzume-llama-3-8B-multilingual-orpo-borda-top75
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
Phi 2 Insurance_Qa Sft Lora
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
SA
Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp
Sakura · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
MI
mistral7b_ocr_to_json_v1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GI
Gille StrangeMerges 32 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
mlfoundations dev llama3 1 8b oh v3.1 wo metamath 40k
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
google-gemma-2b-dequantized
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
Mistrality-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SE
selectorseb s2 oracle trained drone dataset
Selectorseb · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB