arXiv subjects

Yahel Uffenheimer

Publications and source records attributed to Yahel Uffenheimer.

3 recordsLinked to original sources

arXiv2026

Improved Sparse Recovery for Approximate Matrix Multiplication

We present a simple randomized algorithm for approximate matrix multiplication (AMM) whose error scales with the *output* norm $\|AB\|_F$. Given any $n\times n$ matrices $A,B$ and a runtime parameter $r\leq n$, the algorithm produces in $O(n^2(r+\log n))$ time, a matrix $C$ with total squared error $\mathbb{E}[\|C-AB\|_F^2]\le (1-\frac{r}{n})\|AB\|_F^2$, per-entry variance $\|AB\|_F^2/n^2$ and bias $\mathbb{E}[C]=\frac{r}{n}AB$. Alternatively, the algorithm can compute an *unbiased* estimation with expected total squared error $\frac{n}{r}\|{AB}\|_{F}^2$, recovering the state-of-art AMM error obtained by Pagh's TensorSketch algorithm (Pagh, 2013). Our algorithm is a log-factor faster. The key insight in the algorithm is a new variation of pseudo-random rotation of the input matrices (a Fast Hadamard Transform with asymmetric diagonal scaling), which redistributes the Frobenius norm of the *output* $AB$ uniformly across its entries.

cs.DS↗

arXiv2025

(Approximate) Matrix Multiplication via Convolutions

We study the capability of the Fast Fourier Transform (FFT) to accelerate exact and approximate matrix multiplication without using Strassen-like divide-and-conquer. We present a simple exact algorithm running in $O(n^{2.89})$ time, which only sums a few convolutions (FFTs) in $\mathbb{Z}_{m}^{k}$, building on the work of Cohn, Kleinberg, Szegedy and Umans (2005). As a corollary, combining this algorithm with linear sketching breaks the longstanding linear speed-accuracy tradeoff for "combinatorial" approximate matrix multiplication (AMM, Pagh'13, Sarlos'06, Clarkson-Woodruff'13), achieving error $\frac{1}{r^{1.1}}\left\lVert \mathbf{A} \right\rVert_{F}^{2}\left\lVert \mathbf{B}\right\rVert_{F}^{2}$ in $O(rn^{2})$ time, using nothing but FFTs. Motivated by the rich literature for approximating polynomials, our main contribution in this paper is extending the group-theoretic framework of Cohn and Umans (2003) to approximate matrix multiplication (AMM). Specifically, we introduce and study an approximate notion of the Triple Product Property, which in the abelian case is equivalent to finding a Sumset which minimizes (multi-)intersections with an arithmetic progression. We prove tight bounds on this quantity for abelian groups (yielding a simple and practical AMM algorithm via polynomial multiplication), and establish a weaker lower bound for non-abelian groups, extending a lemma of Gowers. Finally, we propose a concrete approach that uses low-degree approximation of multi-variate polynomials for AMM, which we believe will lead to practical, non-asymptotic AMM algorithms in real-world applications, most notably LLM inference.

cs.DS↗

arXiv2025

Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNs

Modern AI relies on huge matrix multiplications (MatMuls), whose computation poses a scalability problem for inference and training. We propose an alternative, GPU native bilinear operator to MatMuls in neural networks, which offers a three-way tradeoff between: speed, accuracy and parameter count. In particular, this operator requires substantially fewer FLOPs to evaluate ($\ll n^3$), yet increases the parameter count compared to MatMul ($\gg n^2$). We call this operator Strassen-Tile (STL). The key idea behind STL is a local learnable change-of-basis, applied on tiles of the weight and activation matrices, followed by an element-wise product between the tiles, implemented simultaneously via MatMul. The key technical question we study is how to optimize the change-of-basis of a given layer, which is a highly non-convex problem. We show that theory-backed initializations (inspired by fast matrix and polynomial multiplication) lead to substantially better accuracy than random SGD initialization. This phenomenon motivates further algorithmic study of STL optimization in DNNs. Our experiments demonstrate that STL can approximate 4x4 MatMul of tiles while reducing FLOPs by a factor of 2.66, and can improve Imagenet-1K accuracy of SoTA T2T-ViT-7 (4.3M parameters) while lowering FLOPs. Even with non-CUDA optimized PyTorch code, STL achieves wall-clock speedups in the compute-bound regime. These results, together with its theoretical grounds, suggest STL as a promising building block for scalable and cost-efficient AI.

cs.LG↗