GGUF quants compared

A quant is the way a model’s weights are stored in a GGUF file. Fewer bits per weight means a smaller file and a little less quality.

The table lists every quant type found in the catalog, with the number of models that offer it, its approximate bits per weight and its size compared with the 16-bit original. Q4_K_M is the most widely offered, by 152,431 models. If you are not sure which to choose, start with Q4_K_M and read GGUF quantization explained.

QuantModelsBits per weightSize vs 16-bitWhat to expect
Q4_K_M152,431~4.9~31%The usual sweet spot between file size and quality, and the quant most people start with.
Q5_K_M118,680~5.7~36%Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.
Q8_0115,478~8.5~53%Close to lossless. It costs roughly twice the memory of a 4-bit file, so it suits cases where you have plenty of VRAM.
Q6_K114,045~6.6~41%Very close to the original quality for most uses, at a clearly larger size than 4-bit.
Q2_K112,959~3~19%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
Q3_K_M109,437~3.9~24%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
Q4_K_S105,482~4.6~29%The usual sweet spot between file size and quality, and the quant most people start with.
Q5_K_S103,670~5.5~34%Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.
Q3_K_S103,315~3.5~22%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
Q3_K_L101,394~4.3~27%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
IQ4_XS98,078~4.3~27%The usual sweet spot between file size and quality, and the quant most people start with.
Q4_064,254~4.5~28%The usual sweet spot between file size and quality, and the quant most people start with.
F1654,977~16~100%The unquantized model. It gives original quality and needs the most memory, so it is mainly for conversion or reference.
IQ3_M48,127~3.7~23%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
IQ3_XS46,251~3.3~21%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
IQ3_S45,608~3.5~22%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
Q4_137,601varies—The usual sweet spot between file size and quality, and the quant most people start with.
IQ4_NL32,702~4.5~28%The usual sweet spot between file size and quality, and the quant most people start with.
IQ2_M28,575~2.7~17%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
IQ3_XXS28,359~3.1~19%A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
IQ2_XXS26,726~2.1~13%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
IQ2_XS26,419~2.3~14%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
IQ2_S26,216~2.5~16%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
IQ1_M25,761~1.8~11%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
IQ1_S25,551~1.6~10%Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
Q5_022,024varies—Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.
Q2_K_S19,749varies—Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
Q5_118,015varies—Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.
Q4_K16,349varies—The usual sweet spot between file size and quality, and the quant most people start with.
Q3_K15,249varies—A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
Q5_K15,203varies—Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.
BF1610,444~16~100%The unquantized model. It gives original quality and needs the most memory, so it is mainly for conversion or reference.
Q3_K_XL2,605varies—A reasonable choice when a 4-bit file will not fit, mostly on larger models. Quality loss is noticeable but often acceptable.
F322,488~32~200%The unquantized model. It gives original quality and needs the most memory, so it is mainly for conversion or reference.
Q2_K_L2,223varies—Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.
Q4_K_L1,901~5~31%The usual sweet spot between file size and quality, and the quant most people start with.
Q6_K_L1,854varies—Very close to the original quality for most uses, at a clearly larger size than 4-bit.
Q5_K_L1,648varies—Closer to the original model than 4-bit, for a moderate increase in size. A good pick when you have memory to spare.
Q4_K_XL1,274~5~31%The usual sweet spot between file size and quality, and the quant most people start with.
Q2_K_XL987varies—Use it only when memory is very tight. Expect a clear drop in quality, which is usually worse on small models and easier to live with on large ones.

Bit widths are approximate averages and differ a little between models.