arXiv subjects
Publications and source records attributed to Arnaud Guillin.
Stochastic gradient descent with momentum is a popular variant of stochastic gradient descent, which has recently been reported to have a close relationship with the underdamped Langevin diffusion. In this paper, we establish a quantitative error estimate between them in the 1-Wasserstein and total variation distances.
MicroLED displays are the result of a complex manufacturing chain. Each stage of this process, if optimized, contributes to achieving the highest levels of final efficiencies. Common works carried out by Pollen Metrology, Aledia, and Universit{é} Clermont-Auvergne led to a generic process optimization workflow. This software solution offers a holistic approach where stages are chained together for gaining a complete optimal solution. This paper highlights key corners of the methodology, validated by the experiments and process experts: data cleaning and multi-objective optimization.
In this paper, we rigorously derive Central Limit Theorems (CLT) for Bayesian two-layerneural networks in the infinite-width limit and trained by variational inference on a regression task. The different networks are trained via different maximization schemes of the regularized evidence lower bound: (i) the idealized case with exact estimation of a multiple Gaussian integral from the reparametrization trick, (ii) a minibatch scheme using Monte Carlo sampling, commonly known as Bayes-by-Backprop, and (iii) a computationally cheaper algorithm named Minimal VI. The latter was recently introduced by leveraging the information obtained at the level of the mean-field limit. Laws of large numbers are already rigorously proven for the three schemes that admits the same asymptotic limit. By deriving CLT, this work shows that the idealized and Bayes-by-Backprop schemes have similar fluctuation behavior, that is different from the Minimal VI one. Numerical experiments then illustrate that the Minimal VI scheme is still more efficient, in spite of bigger variances, thanks to its important gain in computational complexity.
In this paper, we provide bounds in Wasserstein and total variation distances between the distributions of the successive iterates of two functional autoregressive processes with isotropic Gaussian noise of the form $Y_{k+1} = \mathrm{T}_γ(Y_k) + \sqrt{γσ^2} Z_{k+1}$ and $\tilde{Y}_{k+1} = \tilde{\mathrm{T}}_γ(\tilde{Y}_k) + \sqrt{γσ^2} \tilde{Z}_{k+1}$. More precisely, we give non-asymptotic bounds on $ρ(\mathcal{L}(Y_{k}),\mathcal{L}(\tilde{Y}_k))$, where $ρ$ is an appropriate weighted Wasserstein distance or a $V$-distance, uniformly in the parameter $γ$, and on $ρ(π_γ,\tildeπ_γ)$, where $π_γ$ and $\tildeπ_γ$ are the respective stationary measures of the two processes. The class of considered processes encompasses the Euler-Maruyama discretization of Langevin diffusions and its variants. The bounds we derive are of order $γ$ as $γ\to 0$. To obtain our results, we rely on the construction of a discrete sticky Markov chain $(W_k^{(γ)})_{k \in \mathbb{N}}$ which bounds the distance between an appropriate coupling of the two processes. We then establish stability and quantitative convergence results for this process uniformly on $γ$. In addition, we show that it converges in distribution to the continuous sticky process studied in previous work. Finally, we apply our result to Bayesian inference of ODE parameters and numerically illustrate them on two particular problems.
Monte Carlo simulations of systems of particles such as hard spheres or soft spheres with singular kernels can display around a phase transition prohibitively long convergence times when using traditional Hasting-Metropolis reversible schemes. Efficient algorithms known as event-chain Monte Carlo were then developed to reach necessary accelerations. They are based on non-reversible continuous-time Markov processes. Proving invariance and ergodicity for such schemes cannot be done as for discrete-time schemes and a theoretical framework to do so was lacking, impeding the generalisation of ECMC algorithms to more sophisticated systems or processes. In this work, we characterize the Markov processes generated in ECMC as piecewise deterministic Markov processes. It first allows us to propose more general schemes, for instance regarding the direction refreshment. We then prove the invariance of the correct stationary distribution. Finally, we show the ergodicity of the processes in soft- and hard-sphere systems, with a density condition for the latter.
In this article, we adapt the work of Jabin and Wang (2018) to show the first result of uniform in time propagation of chaos for a class of singular interaction kernels. In particular, our models contain the Biot-Savart kernel on the torus and thus the 2D vortex model.
In this article, we focus on two toy models : the Curie-Weiss model and the system of $N$ particles in linear interactions in a double well confining potential. Both models, which have been extensively studied, describe a large system of particles with a mean-field limit that admits a phase transition. We are concerned with the numerical simulation of these particle systems. To deal with the quadratic complexity of the numerical scheme, corresponding to the computation of the $O(N^2)$ interactions per time step, the Random Batch Method (RBM) has been suggested. It consists in randomly (and uniformly) dividing the particles into batches of size $p>1$, and computing the interactions only within each batch, thus reducing the numerical complexity to $O(Np)$ per time step. The convergence of this numerical method has been proved in other works. This work is motivated by the observation that the RBM, via the random constructions of batches, artificially adds noise to the particle system. The goal of this article is to study the effect of this added noise on the phase transition of the nonlinear limit, and more precisely we study the effective dynamics of the two models to show how a phase transition may still be observed with the RBM but at a lower critical temperature.
We provide a rigorous analysis of training by variational inference (VI) of Bayesian neural networks in the two-layer and infinite-width case. We consider a regression problem with a regularized evidence lower bound (ELBO) which is decomposed into the expected log-likelihood of the data and the Kullback-Leibler (KL) divergence between the a priori distribution and the variational posterior. With an appropriate weighting of the KL, we prove a law of large numbers for three different training schemes: (i) the idealized case with exact estimation of a multiple Gaussian integral from the reparametrization trick, (ii) a minibatch scheme using Monte Carlo sampling, commonly known as Bayes by Backprop, and (iii) a new and computationally cheaper algorithm which we introduce as Minimal VI. An important result is that all methods converge to the same mean-field limit. Finally, we illustrate our results numerically and discuss the need for the derivation of a central limit theorem.
In this work, we consider a wide two-layer neural network and study the behavior of its empirical weights under a dynamics set by a stochastic gradient descent along the quadratic loss with mini-batches and noise. Our goal is to prove a trajectorial law of large number as well as a central limit theorem for their evolution. When the noise is scaling as 1/N $β$ and 1/2 < $β$ $\le$ $\infty$, we rigorously derive and generalize the LLN obtained for example in [CRBVE20, MMM19, SS20b]. When 3/4 < $β$ $\le$ $\infty$, we also generalize the CLT (see also [SS20a]) and further exhibit the effect of mini-batching on the asymptotic variance which leads the fluctuations. The case $β$ = 3/4 is trickier and we give an example showing the divergence with time of the variance thus establishing the instability of the predictions of the neural network in this case. It is illustrated by simple numerical examples.
We develop a new approach to study the long time behaviour of solutions to nonlinear stochastic differential equations in the sense of McKean, as well as propagation of chaos for the corresponding mean-field particle system approximations. Our approach is based on a sticky coupling between two solutions to the equation. We show that the distance process between the two copies is dominated by a solution to a one-dimensional nonlinear stochastic differential equation with a sticky boundary at zero. This new class of equations is then analyzed carefully. In particular, we show that the dominating equation has a phase transition. In the regime where the Dirac measure at zero is the only invariant probability measure, we prove exponential convergence to equilibrium both for the one-dimensional equation, and for the original nonlinear SDE. Similarly, propagation of chaos is shown by a componentwise sticky coupling and comparison with a system of one dimensional nonlinear SDEs with sticky boundaries at zero. The approach applies to equations without confinement potential and to interaction terms that are not of gradient type.
In this article, we prove the first quantitative uniform in time propagation of chaos for a class of systems of particles in singular repulsive interaction in dimension one that contains the Dyson Brownian motion. We start by establishing existence and uniqueness for the Riesz gases, before proving propagation of chaos with an original approach to the problem, namely coupling with a Cauchy sequence type argument. We also give a general argument to turn a result of weak propagation of chaos into a strong and uniform in time result using the long time behavior and some bounds on moments, in particular enabling us to get a uniform in time version of the result of Cépa-Lépingle.
As the title indicates this paper will describe several extensions and applications of the $Γ$2 integrated criterion introduced by M. Ledoux following ideas of B. Hellffer. We introduce general weak versions and show that they are equivalent to the weak Poincar{é} inequalities introduced by M. R{ö}ckner and F. Y. Wang. We also discuss special weak versions appropriate to the study of log-concave measures and log-concave perturbations of product measures.
Among all generalized Ornstein-Uhlenbeck processes which sample the same invariant measure and for which the same amount of randomness (a $N$-dimensional Brownian motion) is injected in the system, we prove that the asymptotic rate of convergence is maximized by a non-reversible hypoelliptic one.
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which implies quantitative bounds on the total variation between the marginal distributions of the two processes. Finally two results are established regarding the invariant measures of PDMPs. A practical condition to show that a probability measure is invariant for the associated PDMP semi-group is presented. In a second time, a bound on the invariant probability measures in $V$-norm of two PDMPs following the same differential flow is established. This last result is then applied to study the asymptotic bias of some non-exact PDMP MCMC methods.
We prove the existence of a contraction rate for Vlasov-Fokker-Planck equation in Wasserstein distance, provided the interaction potential is (locally) Lipschitz continuous and the confining potential is both Lipschitz continuous and greater than a quadratic function, thus requiring no convexity conditions. Our strategy relies on coupling methods suggested by A. Eberle adapted to the kinetic setting enabling also to obtain uniform in time propagation of chaos in a non convex setting.
We study functional inequalities (Poincaré, Cheeger, log-Sobolev) for probability measures obtained as perturbations. Several explicit results for general measures as well as log-concave distributions are given.The initial goal of this work was to obtain explicit bounds on the constants in view of statistical applications for instance. These results are then applied to the Langevin Monte-Carlo method used in statistics in order to compute Bayesian estimators.
Combining the results of [14] and [10], the trend to equilibrium in large time is studied for a large particle system associated to a Vlasov-Fokker-Planck equation. Under some conditions (that allow non-convex confining potentials) the convergence rate is proven to be independent from the number of particles. From this are derived uniform in time propagation of chaos estimates and an exponentially fast convergence for the nonlinear equation itself.
We study Poincar{é} inequalities and long-time behavior for diffusion processes on R^n under a variable curvature lower bound, in the sense of Bakry-Emery. We derive various estimates on the rate of convergence to equilibrium in L^1 optimal transport distance, as well as bounds on the constant in the Poincar{é} inequality in several situations of interest, including some where curvature may be negative. In particular, we prove a self-improvement of the Bakry-Emery estimate for Poincar{é} inequalities when curvature is positive but not constant.