Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution

Abstract

Rotation-based quantization -- multiplying weights and activations by a Hadamard matrix before quantizing -- is now standard, but why it works is explained informally, via incoherence, the central limit theorem, or kurtosis, and rarely tested directly. We ask which property of the post-rotation distribution drives the gain, using paired hypothesis tests on the weights and activations of an image super-resolution transformer. Three candidate properties separate cleanly. Range reduction replicates under every orthogonal transform we tried -- Sylvester Hadamard, QuIP#-Paley, DCT-II, and random orthogonal matrices -- and transfers to a state-space backbone and to CNN weights. Improved normality also replicates. The increased mass near zero, by contrast, is significant only under the zero-padded Sylvester construction and disappears under unpadded transforms, so we attribute it to the padding, not the transform. The end task follows the same pattern: over a grid of three scale factors, three bitwidths and five benchmarks, rotation improves PSNR in all 45 configurations against an otherwise identical un-rotated quantizer at the same nominal bitwidth (the rotated arm carries 0.267 bits/parameter of Sylvester padding at 4 bits and stays ahead in every cell when charged for it), while substituting DCT-II or a random orthogonal matrix costs only 0.06-0.07 dB, less than the padding the Hadamard is charged for. The choice of kernel is therefore an implementation decision: the Hadamard's advantage is its +/-1 entries and O(n log n) butterflies. We package the recipe as CompSRT, which improves on 2DQuant in 44 of 45 configurations at equal nominal bitwidth -- charged for its padding that edge holds at 3 and 2 bits but not at 4. CondiQuant remains ahead at every operating point by 0.03-1.40 dB, which we report.