Storing a model's weights at lower numerical precision, such as 8-bit integers, so it needs less memory and runs faster with little loss in quality.
Model Quantization makes it practical to deploy Large Language Models at the edge or on constrained On-Premises hardware. Compressing the weights to lower precision, often with techniques such as GPTQ or AWQ, lets the model use a fraction of the VRAM while keeping accuracy close to the original.
See also On-Premises, Neural Network, Private AI, Large Language Model, Fine-Tuning and Model Inference.