Algebraic Invariants of Lightning Self-Attention

Abstract

We study the polynomial coefficients of lightning self-attention as coordinates of an algebraic variety. In the single-token case, we characterize the coefficient variety as a rank-constrained Chow-type variety and derive algebraic equations from this description. For multiple tokens, linear relations reduce the nonlinear geometry to coefficients involving interactions between distinct tokens. We characterize the resulting variety by a common linear factor together with a low-rank condition on the remaining factors. This structure yields explicit families of determinantal, Veronese-type, and Sylvester resultant-based invariants. In the rank-one case, natural pencil and flattening equations define the variety set-theoretically. We complement these results with computations in small dimensions, where the theoretical constructions recover the generators of the defining ideals. Overall, the paper gives a structural description of the algebraic constraints imposed on polynomial coefficient arrays by lightning self-attention and provides explicit certificates of non-realizability.