Dynamical mean-field analysis of adaptive Langevin diffusions: Propagation-of-chaos and convergence of the linear response

Abstract

Motivated by an application to empirical Bayes learning in high-dimensional regression, we study a class of Langevin diffusions in a system with random disorder, where the drift coefficient is driven by a parameter that continuously adapts to the empirical distribution of the realized process up to the current time. The resulting dynamics take the form of a stochastic interacting particle system having both a McKean-Vlasov type interaction and a pairwise interaction defined by the random disorder. We prove a propagation-of-chaos result, showing that in the large system limit over dimension-independent time horizons, the empirical distribution of sample paths of the Langevin process converges to a deterministic limit law that is described by dynamical mean-field theory. This law is characterized by a system of dynamical fixed-point equations for the limit of the drift parameter and for the correlation and response kernels of the limiting dynamics. Using a dynamical cavity argument, we verify that these correlation and response kernels arise as the asymptotic limits of the averaged correlation and linear response functions of single coordinates of the system. These results enable an asymptotic analysis of an empirical Bayes Langevin dynamics procedure for learning an unknown prior parameter in a linear regression model, which we develop in a companion paper.