GGUF guides
Short guides that explain the numbers on the model pages.
GGUF quantization explained
What Q4_K_M, Q5_K_M, Q8_0 and IQ quants mean, how much quality you lose, and which GGUF quant to choose for your memory.
How much VRAM do you need to run a local LLM?
A practical guide to GGUF memory needs: file size, KV cache, context length and runtime overhead, with a table of what fits 4 to 48 GB.