arXiv · 2502.05074
Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
Abstract
We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide variety of high-dimensional linear models trained with stochastic gradient descent. This includes high-dimensional linear regression, kernel regression, and linear random feature models. Our results include previously known asymptotics as well as novel ones.
Explore related subjects
Keep this discovery
Alexander Atanasov, Blake Bordelon, Jacob A. Zavatone-Veth, Courtney Paquette, Cengiz Pehlevan. 2025-02-07. Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models. https://doi.org/10.4310/atmp.260412235902
Cite the original work for its findings. Save a collection to share your selection of sources.