Quantization is the technical decision with the greatest impact on how much hardware you need to run a language model. Reducing the numerical precision used to store the weights can cut VRAM requirements by up to 75%, but the choice comes at a cost: there is always a trade-off between memory, speed and response quality.
What it means to “quantize” a model
A trained model stores its parameters as floating-point numbers, usually in FP32 or FP16. Quantization represents those same parameters with fewer bits — for example, 8-bit or 4-bit integers — reducing the memory footprint in exchange for a loss of numerical precision that, in most cases, has a limited effect on output quality.
Comparing the most common precisions
| Precision | Bytes/parameter | Weight memory | Usage criterion |
|---|---|---|---|
| FP16 / BF16 | 2 | 100% (baseline) | Baseline for weight precision. |
| INT8 | 1 | ≈50% | Validate quality and acceleration in the chosen engine. |
| INT4 (GPTQ/AWQ) | 0.5 | ≈25% | Saves more memory; requires evaluation on the specific task. |
The real trade-off: more than memory
Reducing precision saves VRAM, but also affects:
- Inference speed. In general, quantization speeds up inference because less data has to move between memory and compute units — although the exact effect depends on the hardware and inference engine.
- Response quality. Degradation is neither linear nor uniform: general knowledge tasks often tolerate INT8 and even INT4 well, while mathematical reasoning and code generation are more sensitive to precision loss.
- Hardware compatibility. Not all GPUs accelerate each quantization type equally well; some architectures have native INT8 support that others lack.
How to decide which precision to use
The right question is not “which quantization is best?” but “how much quality can I afford to trade for the hardware I have?”. As a practical reference:
- If you have enough VRAM for FP16 and quality is critical (decision support or regulated environments), stay with FP16.
- If you need to serve more concurrent requests or run a larger model on the same hardware, INT8 is often a reasonable balance.
- INT4 makes sense when hardware is a hard constraint (a single mid-range GPU or an edge environment) and you can validate that the specific task tolerates the loss of precision.
In every case, the only reliable way to know is to measure response quality for your own use case before deploying to production — generic benchmarks offer guidance, but cannot replace a test with your own data.
How this affects sizing
The chosen precision is the first variable we adjust in our guide to how much VRAM an LLM needs, because it directly changes the size of the weights in memory. You can compare the actual impact for your specific model with the LLM Calculator, which lets you switch between precisions and instantly see the resulting VRAM margin.