Constraint-Free Post-Training Quantization for Low-Bit Transformers-raw data

Quantized low-bit Transformers demonstrate a competitive edge by requiring less storage and delivering significantly accelerated inference. However, mainstream methods are limited by strict constraints such as fixed quantization intervals, thus leading to large quantization errors (i.e., clipping and rounding errors) and substantial performance degradation.

To address this issue, we propose Constraint-Free Post-training Quantization (CFQuant), which enables flexible quantization by modeling activation distributions through three key steps. First, CFQuant adaptively estimates the density of activation distributions and minimizes quantization errors through iterative search during calibration. Second, an Efficient Scale-shift Algorithm (ESA) is designed to dynamically adjust activation distributions, reducing the distribution shift between the calibration and inference stages.