Quantization Lab
Compress weights into low-bit formats and visualize the accuracy trade-offs.
Modern models are large and memory-heavy. Quantization reduces each weight from 32-bit floats to fewer bits (often 8, 4, or even 1), shrinking the model and speeding up inference.
The core trade-off is accuracy versus efficiency. With careful calibration, low-bit models can retain most performance while running faster on commodity hardware.
Uniform Symmetric: scales weights around zero; simple and hardware-friendly.
Uniform Asymmetric: shifts the range to better fit non-zero distributions.
Dynamic Range (per-row): uses a separate scale for each row, improving fidelity on heterogeneous matrices.
Log / Binary / Ternary: aggressive compression for extreme efficiency, at the cost of distortion.
- Generate a random matrix or set its density.
- Select a quantization mode and bit-width.
- Apply quantization and inspect MSE, PSNR, and the error heatmap.