Ozaki Scheme II Is Fast on CPUs Too: Multiple-Precision Matrix Multiplication on Intel AMX-INT8 and Arm SVE2-i8mm

Abstract

We implement Ozaki Scheme II (residue number system + Chinese remainder theorem), which reduces multiple-precision dense matrix multiplication to a sequence of low-precision, high-throughput integer or floating-point GEMMs on CPUs. Two backends are built on top of a shared CRT reconstruction stage: (a) exact INT8 x INT8 -> INT32 tile products on Intel AMX, and (b) binary64 DGEMM, the CPU construction of the original Ozaki Scheme II paper. On a two-socket Xeon Gold 6526Y (Emerald Rapids, 32 cores), we evaluate significand precisions of 53-2048 bits and matrix dimensions N = 256-8192. The results are always within 1 ulp of a high-precision MPFR reference (essentially correctly rounded), while running up to 167x faster than a naive MPFR matrix product, up to 588x faster than BNCmatmul's Strassen multiplication, and 9-78x faster than Ozaki Scheme I (FP64 slicing + OpenBLAS DGEMM). The break-even point between the two backends is approximately N = 2048: below it, the binary64 backend wins thanks to its smaller number of moduli; above it, the AMX-INT8 backend wins as the GEMMs dominate. We further port the implementation to AArch64 (NVIDIA GB10: Cortex-X925 x 10 + Cortex-A725 x 10). Since this machine lacks SME/SME2, the INT8 kernel uses the SMMLA matrix-product instruction of the SVE2 i8mm extension. We obtain an exact INT8 GEMM sustaining 6.5 TOPS and, still within 1 ulp across all conditions, speedups of 14-89x over BNCmatmul's Ozaki Scheme I (OpenBLAS-linked routine) and 6-19x over a fairness-adjusted OzI-best variant. The paper also includes a tutorial introduction to Ozaki Scheme II (Section "Introduction to Ozaki Scheme II") and a quantitative explanation of why this seemingly GPU-specific technique is fast on CPUs as well.