arXiv subjects

Wuchen Li

Publications and source records attributed to Wuchen Li.

At least 73 records · Page 4Linked to original sources

arXiv2022

Accelerated Information Gradient flow

We present a framework for Nesterov's accelerated gradient flows in probability space to design efficient mean-field Markov chain Monte Carlo (MCMC) algorithms for Bayesian inverse problems. Here four examples of information metrics are considered, including Fisher-Rao metric, Wasserstein-2 metric, Kalman-Wasserstein metric and Stein metric. For both Fisher-Rao and Wasserstein-2 metrics, we prove convergence properties of accelerated gradient flows. In implementations, we propose a sampling-efficient discrete-time algorithm for Wasserstein-2, Kalman-Wasserstein and Stein accelerated gradient flows with a restart technique. We also formulate a kernel bandwidth selection method, which learns the gradient of logarithm of density from Brownian-motion samples. Numerical experiments, including Bayesian logistic regression and Bayesian neural network, show the strength of the proposed methods compared with state-of-the-art algorithms.

math.OC↗

arXiv2022

Neural Parametric Fokker-Planck Equations

In this paper, we develop and analyze numerical methods for high dimensional Fokker-Planck equations by leveraging generative models from deep learning. Our starting point is a formulation of the Fokker-Planck equation as a system of ordinary differential equations (ODEs) on finite-dimensional parameter space with the parameters inherited from generative models such as normalizing flows. We call such ODEs neural parametric Fokker-Planck equations. The fact that the Fokker-Planck equation can be viewed as the $L^2$-Wasserstein gradient flow of Kullback-Leibler (KL) divergence allows us to derive the ODEs as the constrained $L^2$-Wasserstein gradient flow of KL divergence on the set of probability densities generated by neural networks. For numerical computation, we design a variational semi-implicit scheme for the time discretization of the proposed ODE. Such an algorithm is sampling-based, which can readily handle the Fokker-Planck equations in higher dimensional spaces. Moreover, we also establish bounds for the asymptotic convergence analysis of the neural parametric Fokker-Planck equation as well as the error analysis for both the continuous and discrete versions. Several numerical examples are provided to illustrate the performance of the proposed algorithms and analysis.

math.NA↗

arXiv2022

Computational Mean-field Games on Manifolds

Conventional Mean-field games/control study the behavior of a large number of rational agents moving in the Euclidean spaces. In this work, we explore the mean-field games on Riemannian manifolds. We formulate the mean-field game Nash Equilibrium on manifolds. We also establish the equivalence between the PDE system and the optimality conditions of the associated variational form on manifolds. Based on the triangular mesh representation of two-dimensional manifolds, we design a proximal gradient method for variational mean-field games. Our comprehensive numerical experiments on various manifolds illustrate the effectiveness and flexibility of the proposed model and numerical methods.

math.OC↗

arXiv2022

Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization

The computation of Wasserstein gradient direction is essential for posterior sampling problems and scientific computing. The approximation of the Wasserstein gradient with finite samples requires solving a variational problem. We study the variational problem in the family of two-layer networks with squared-ReLU activations, towards which we derive a semi-definite programming (SDP) relaxation. This SDP can be viewed as an approximation of the Wasserstein gradient in a broader function family including two-layer networks. By solving the convex SDP, we obtain the optimal approximation of the Wasserstein gradient direction in this class of functions. Numerical experiments including PDE-constrained Bayesian inference and parameter estimation in COVID-19 modeling demonstrate the effectiveness of the proposed method.

cs.LG↗

arXiv2022

Mean field information Hessian matrices on graphs

We derive mean-field information Hessian matrices on finite graphs. The "information" refers to entropy functions on the probability simplex. And the "mean-field" means nonlinear weight functions of probabilities supported on graphs. These two concepts define a mean-field optimal transport type metric. In this metric space, we first derive Hessian matrices of energies on graphs, including linear, interaction energies, entropies. We name their smallest eigenvalues as mean-field Ricci curvature bounds on graphs. We next provide examples on two-point spaces and graph products. We last present several applications of the proposed matrices. E.g., we prove discrete Costa's entropy power inequalities on a two-point space.

math.CO↗

arXiv2022

Mean field Kuramoto models on graphs

One of a classical synchronization model is the Kuramoto model. We propose both first and second order Kuramoto dynamical models on graphs using discrete optimal transport dynamics. We analyze the synchronization behaviors for some examples of Kuramoto models on graphs. We also provide a generalized Hopf-Cole transformation for discrete optimal transport systems. Focus on the two points graph, we derive analytical formulas of the Kuramoto dynamics with various potential induced from entropy functionals. Several numerical examples for the Kuramoto model on general graphs are presented.

math.DS↗

arXiv2022

Controlling conservation laws II: compressible Navier-Stokes equations

We propose, study, and compute solutions to a class of optimal control problems for hyperbolic systems of conservation laws and their viscous regularization. We take barotropic compressible Navier--Stokes equations (BNS) as a canonical example. We first apply the entropy--entropy flux--metric condition for BNS. We select an entropy function and rewrite BNS to a summation of flux and metric gradient of entropy. We then develop a metric variational problem for BNS, whose critical points form a primal-dual BNS system. We design a finite difference scheme for the variational system. The numerical approximations of conservation laws are implicit in time. We solve the variational problem with an algorithm inspired by the primal-dual hybrid gradient method. This includes a new method for solving implicit time approximations for conservation laws, which seems to be unconditionally stable. Several numerical examples are presented to demonstrate the effectiveness of the proposed algorithm.

math.OC↗

arXiv2021

Controlling conservation laws I: entropy-entropy flux

We study a class of variational problems for regularized conservation laws with Lax's entropy-entropy flux pairs. We first introduce a modified optimal transport space based on conservation laws with diffusion. Using this space, we demonstrate that conservation laws with diffusion are "flux--gradient flows". We next construct variational problems for these flows, for which we derive dual PDE systems for regularized conservation laws. Several examples, including traffic flow and Burgers' equation, are presented. Incorporating both primal-dual algorithms and monotone schemes, we successfully compute the control of conservation laws.

math.AP↗

arXiv2021

Tracial smooth functions of non-commuting variables and the free Wasserstein manifold

We formulate a free probabilistic analog of the Wasserstein manifold on $\mathbb{R}^d$ (the formal Riemannian manifold of smooth probability densities on $\mathbb{R}^d$), and we use it to study smooth non-commutative transport of measure. The points of the free Wasserstein manifold $\mathscr{W}(\mathbb{R}^{*d})$ are smooth tracial non-commutative functions $V$ with quadratic growth at $\infty$, which correspond to minus the log-density in the classical setting. The space of smooth tracial non-commutative functions used here is a new one whose definition and basic properties we develop in the paper; they are scalar-valued functions of self-adjoint $d$-tuples from arbitrary tracial von Neumann algebras that can be approximated by trace polynomials. The space of non-commutative diffeomorphisms $\mathscr{D}(\mathbb{R}^{*d})$ acts on $\mathscr{W}(\mathbb{R}^{*d})$ by transport, and the basic relationship between tangent vectors for $\mathscr{D}(\mathbb{R}^{*d})$ and tangent vectors for $\mathscr{W}(\mathbb{R}^{*d})$ is described using the Laplacian $L_V$ associated to $V$ and its pseudo-inverse $Ψ_V$ (when defined). Following similar arguments to arXiv:1204.2182, arXiv:1701.00132, and arXiv:1906.10051 in the new setting, we give a rigorous proof for the existence of smooth transport along any path $t \mapsto V_t$ when $V$ is sufficiently close $(1/2) \sum_j \operatorname{tr}(x_j^2)$, as well as smooth triangular transport.

math.OA↗

arXiv2021

Hessian metric via transport information geometry

We propose to study the Hessian metric of a functional on the space of probability measures endowed with the Wasserstein $2$-metric. We name it transport Hessian metric, which contains and extends the classical Wasserstein-$2$ metric. We formulate several dynamical systems associated with transport Hessian metrics. Several connections between transport Hessian metrics and mathematical physics equations are discovered. E.g., the transport Hessian gradient flow, including Newton's flow, formulates a mean-field kernel Stein variational gradient flow; The transport Hessian Hamiltonian flow of Boltzmann-Shannon entropy forms the Shallow water equation; The transport Hessian gradient flow of Fisher information leads to the heat equation. Several examples and closed-form solutions for transport Hessian distances are presented.

math.DG↗

arXiv2021

Tropical Optimal Transport and Wasserstein Distances

We study the problem of optimal transport in tropical geometry and define the Wasserstein-$p$ distances in the continuous metric measure space setting of the tropical projective torus. We specify the tropical metric -- a combinatorial metric that has been used to study of the tropical geometric space of phylogenetic trees -- as the ground metric and study the cases of $p=1,2$ in detail. The case of $p=1$ gives an efficient computation of the infinitely-many geodesics on the tropical projective torus, while the case of $p=2$ gives a form for Fréchet means and a general inner product structure. Our results also provide theoretical foundations for geometric insight a statistical framework in a tropical geometric setting. We construct explicit algorithms for the computation of the tropical Wasserstein-1 and 2 distances and prove their convergence. Our results provide the first study of the Wasserstein distances and optimal transport in tropical geometry. Several numerical examples are provided.

math.OC↗

arXiv2021

Natural gradient via optimal transport

We study a natural Wasserstein gradient flow on manifolds of probability distributions with discrete sample spaces. We derive the Riemannian structure for the probability simplex from the dynamical formulation of the Wasserstein distance on a weighted graph. We pull back the geometric structure to the parameter space of any given probability model, which allows us to define a natural gradient flow there. In contrast to the natural Fisher-Rao gradient, the natural Wasserstein gradient incorporates a ground metric on sample space. We illustrate the analysis of elementary exponential family examples and demonstrate an application of the Wasserstein natural gradient to maximum likelihood estimation.

math.OC↗

arXiv2021

A Fast Proximal Gradient Method and Convergence Analysis for Dynamic Mean Field Planning

In this paper, we propose an efficient and flexible algorithm to solve dynamic mean-field planning problems based on an accelerated proximal gradient method. Besides an easy-to-implement gradient descent step in this algorithm, a crucial projection step becomes solving an elliptic equation whose solution can be obtained by conventional methods efficiently. By induction on iterations used in the algorithm, we theoretically show that the proposed discrete solution converges to the underlying continuous solution as the grid size increases. Furthermore, we generalize our algorithm to mean-field game problems and accelerate it using multilevel and multigrid strategies. We conduct comprehensive numerical experiments to confirm the convergence analysis of the proposed algorithm, to show its efficiency and mass preservation property by comparing it with state-of-the-art methods, and to illustrates its flexibility for handling various mean-field variational problems.

math.OC↗

arXiv2021

Transport information Hessian distances

We formulate closed-form Hessian distances of information entropies in one-dimensional probability density space embedded with the L2-Wasserstein metric.

math.MG↗

arXiv2021

Projected Wasserstein gradient descent for high-dimensional Bayesian inference

We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. The underlying density function of a particle system of WGD is approximated by kernel density estimation (KDE), which faces the long-standing curse of dimensionality. We overcome this challenge by exploiting the intrinsic low-rank structure in the difference between the posterior and prior distributions. The parameters are projected into a low-dimensional subspace to alleviate the approximation error of KDE in high dimensions. We formulate a projected Wasserstein gradient flow and analyze its convergence property under mild assumptions. Several numerical experiments illustrate the accuracy, convergence, and complexity scalability of pWGD with respect to parameter dimension, sample size, and processor cores.

cs.LG↗

arXiv2021

Wasserstein Proximal of GANs

We introduce a new method for training generative adversarial networks by applying the Wasserstein-2 metric proximal on the generators. The approach is based on Wasserstein information geometry. It defines a parametrization invariant natural gradient by pulling back optimal transport structures from probability space to parameter space. We obtain easy-to-implement iterative regularizers for the parameter updates of implicit deep generative models. Our experiments demonstrate that this method improves the speed and stability of training in terms of wall-clock time and Fréchet Inception Distance.

cs.LG↗

arXiv2020

Ricci curvature for parametric statistics via optimal transport

We elaborate the notion of a Ricci curvature lower bound for parametrized statistical models. Following the seminal ideas of Lott-Strum-Villani, we define this notion based on the geodesic convexity of the Kullback-Leibler divergence in a Wasserstein statistical manifold, that is, a manifold of probability distributions endowed with a Wasserstein metric tensor structure. Within these definitions, the Ricci curvature is related to both, information geometry and Wasserstein geometry. These definitions allow us to formulate bounds on the convergence rate of Wasserstein gradient flows and information functional inequalities in parameter space. We discuss examples of Ricci curvature lower bounds and convergence rates in exponential family models.

math.ST↗

arXiv2020

Entropy dissipation via Information Gamma calculus: Non-reversible stochastic differential equations

We formulate explicit bounds to guarantee the exponential dissipation for some non-gradient stochastic differential equations towards their invariant distributions. Our method extends the connection between Gamma calculus and Hessian operators in $L^2$--Wasserstein space. In details, we apply Lyapunov methods in the space of probabilities, where the Lyapunov functional is chosen as the relative Fisher information. We derive the Fisher information induced Gamma calculus to handle non-gradient drift vector fields. We obtain the explicit dissipation bound in terms of $L_1$ distance and formulate the non-reversible Poincar{é} inequality. An analytical example is provided for a non-reversible Langevin dynamic.

math.PR↗