arXiv subjects

Xianjin Yang

Publications and source records attributed to Xianjin Yang.

16 recordsLinked to original sources

arXiv2026

A Globally Convergent Flow for Time-Dependent Mean Field Games and a Solver-Agnostic Framework for Inverse Problems

Mean field games (MFGs) describe the limiting behavior of large populations of strategically interacting agents. This paper addresses two numerical challenges for MFGs: globally convergent forward solvers and solver-agnostic methods for inverse problems. For the forward problem, we extend the Hessian--Riemannian flow (HRF), previously developed for stationary MFGs, to time-dependent MFGs. We first discretize the system in space and time and then construct the flow directly on the resulting finite-dimensional problem. The proposed flow exploits Lasry--Lions monotonicity, preserves the initial density and terminal value function, and maintains positivity and mass of the density. Under standard assumptions, we prove global convergence of the HRF and show how to recover a solution of the full discretized time-dependent MFG system from its limit. For the inverse problem, we formulate parameter estimation as a bilevel problem in which the outer problem updates unknown coefficients and the inner problem solves the discretized MFG system. Gradients of the outer objective are obtained by differentiating the discretized MFG system at the inner solution, rather than differentiating through the iterations of a particular forward solver. This yields a solver-agnostic framework with adjoint-based gradient descent and Gauss--Newton acceleration. Numerical experiments on stationary and time-dependent MFGs demonstrate the effectiveness of the proposed methods.

math.OC↗

arXiv2025

Solving Functional PDEs with Gaussian Processes and Applications to Functional Renormalization Group Equations

We present an operator learning framework for solving non-perturbative functional renormalization group equations, which are integro-differential equations defined on functionals. Our proposed approach uses Gaussian process operator learning to construct a flexible functional representation formulated directly on function space, making it independent of a particular equation or discretization. Our method is flexible, and can apply to a broad range of functional differential equations while still allowing for the incorporation of physical priors in either the prior mean or the kernel design. We demonstrate the performance of our method on several relevant equations, such as the Wetterich and Wilson--Polchinski equations, showing that it achieves equal or better performance than existing approximations such as the local-potential approximation, while being significantly more flexible. In particular, our method can handle non-constant fields, making it promising for the study of more complex field configurations, such as instantons.

cs.LG↗

arXiv2025

Operator Learning at Machine Precision

Neural operator learning methods have garnered significant attention in scientific computing for their ability to approximate infinite-dimensional operators. However, increasing their complexity often fails to substantially improve their accuracy, leaving them on par with much simpler approaches such as kernel methods and more traditional reduced-order models. In this article, we set out to address this shortcoming and introduce CHONKNORIS (Cholesky Newton--Kantorovich Neural Operator Residual Iterative System), an operator learning paradigm that can achieve machine precision. CHONKNORIS draws on numerical analysis: many nonlinear forward and inverse PDE problems are solvable by Newton-type methods. Rather than regressing the solution operator itself, our method regresses the Cholesky factors of the elliptic operator associated with Tikhonov-regularized Newton--Kantorovich updates. The resulting unrolled iteration yields a neural architecture whose machine-precision behavior follows from achieving a contractive map, requiring far lower accuracy than end-to-end approximation of the solution operator. We benchmark CHONKNORIS on a range of nonlinear forward and inverse problems, including a nonlinear elliptic equation, Burgers' equation, a nonlinear Darcy flow problem, the Calderón problem, an inverse wave scattering problem, and a problem from seismic imaging. We also present theoretical guarantees for the convergence of CHONKNORIS in terms of the accuracy of the emulated Cholesky factors. Additionally, we introduce a foundation model variant, FONKNORIS (Foundation Newton--Kantorovich Neural Operator Residual Iterative System), which aggregates multiple pre-trained CHONKNORIS experts for diverse PDEs to emulate the solution map of a novel nonlinear PDE. Our FONKNORIS model is able to accurately solve unseen nonlinear PDEs such as the Klein--Gordon and Sine--Gordon equations.

cs.LG↗

arXiv2025

Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dots

Most problems within and beyond the scientific domain can be framed into one of the following three levels of complexity of function approximation. Type 1: Approximate an unknown function given input/output data. Type 2: Consider a collection of variables and functions, some of which are unknown, indexed by the nodes and hyperedges of a hypergraph (a generalized graph where edges can connect more than two vertices). Given partial observations of the variables of the hypergraph (satisfying the functional dependencies imposed by its structure), approximate all the unobserved variables and unknown functions. Type 3: Expanding on Type 2, if the hypergraph structure itself is unknown, use partial observations of the variables of the hypergraph to discover its structure and approximate its unknown functions. These hypergraphs offer a natural platform for organizing, communicating, and processing computational knowledge. While most scientific problems can be framed as the data-driven discovery of unknown functions in a computational hypergraph whose structure is known (Type 2), many require the data-driven discovery of the structure (connectivity) of the hypergraph itself (Type 3). We introduce an interpretable Gaussian Process (GP) framework for such (Type 3) problems that does not require randomization of the data, access to or control over its sampling, or sparsity of the unknown functions in a known or learned basis. Its polynomial complexity, which contrasts sharply with the super-exponential complexity of causal inference methods, is enabled by the nonlinear ANOVA capabilities of GPs used as a sensing mechanism.

cs.LG↗

arXiv2025

Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes

Methods for solving scientific computing and inference problems, such as kernel- and neural network-based approaches for partial differential equations (PDEs), inverse problems, and supervised learning tasks, depend crucially on the choice of hyperparameters. Specifically, the efficacy of such methods, and in particular their accuracy, stability, and generalization properties, strongly depends on the choice of hyperparameters. While bilevel optimization offers a principled framework for hyperparameter tuning, its nested optimization structure can be computationally demanding, especially in PDE-constrained contexts. In this paper, we propose an efficient strategy for hyperparameter optimization within the bilevel framework by employing a Gauss-Newton linearization of the inner optimization step. Our approach provides closed-form updates, eliminating the need for repeated costly PDE solves. As a result, each iteration of the outer loop reduces to a single linearized PDE solve, followed by explicit gradient-based hyperparameter updates. We demonstrate the effectiveness of the proposed method through Gaussian process models applied to nonlinear PDEs and to PDE inverse problems. Extensive numerical experiments highlight substantial improvements in accuracy and robustness compared to conventional random hyperparameter initialization. In particular, experiments with additive kernels and neural network-parameterized deep kernels demonstrate the method's scalability and effectiveness for high-dimensional hyperparameter optimization.

stat.ML↗

arXiv2025

Gaussian process policy iteration with additive Schwarz acceleration for forward and inverse HJB and mean field game problems

In this paper, we propose a Gaussian Process (GP)-based policy iteration framework for addressing both forward and inverse problems in Hamilton--Jacobi--Bellman (HJB) equations and mean field games (MFGs). Policy iteration is formulated as an alternating procedure between evaluating the value function under a fixed control policy and improving the policy. In our approach, we model the unknown fields using GPs within a policy-iteration framework that converts the nonlinear system into a sequence of linear PDE subproblems. Then, leveraging the linear structure, the updates for the value function and, in the MFG setting, the population density admit explicit representer formulas under linear PDE collocation constraints. The policy is subsequently updated pointwise via a Legendre transform step, which involves a low-dimensional maximization over the control variable. This maximization is explicit for standard quadratic costs. For smooth, strictly convex costs, this pointwise maximization is solved through its first-order optimality condition, whereas in constrained or non-smooth cases, it becomes a low-dimensional constrained maximization problem. To improve convergence, we incorporate the additive Schwarz acceleration as a preconditioning step following each policy update. Numerical experiments demonstrate the effectiveness of the Schwarz acceleration in improving computational efficiency.

cs.LG↗

arXiv2025

Learning Surrogate Potential Mean Field Games via Gaussian Processes: A Data-Driven Approach to Ill-Posed Inverse Problems

Mean field games (MFGs) describe the collective behavior of large populations of interacting agents. In this work, we tackle ill-posed inverse problems in potential MFGs, aiming to recover the agents' population, momentum, and environmental setup from limited, noisy measurements and partial observations. These problems are ill-posed because multiple MFG configurations can explain the same data, or different parameters can yield nearly identical observations. Nonetheless, they remain crucial in practice for real-world scenarios where data are inherently sparse or noisy, or where the MFG structure is not fully determined. Our focus is on finding surrogate MFGs that accurately reproduce the observed data despite these challenges. We propose two Gaussian process (GP)-based frameworks: an inf-sup formulation and a bilevel approach. The choice between them depends on whether the unknown parameters introduce concavity in the objective. In the inf-sup framework, we use the linearity of GPs and their parameterization structure to maintain convex-concave properties, allowing us to apply standard convex optimization algorithms. In the bilevel framework, we employ a gradient-descent-based algorithm and introduce two methods for computing the outer gradient. The first method leverages an existing solver for the inner potential MFG and applies automatic differentiation, while the second adopts an adjoint-based strategy that computes the outer gradient independently of the inner solver. Our numerical experiments show that when sufficient prior information is available, the unknown parameters can be accurately recovered. Otherwise, if prior information is limited, the inverse problem is ill-posed, but our frameworks can still produce surrogate MFG models that closely match observed data.

cs.LG↗

arXiv2025

Solving Roughly Forced Nonlinear PDEs via Misspecified Kernel Methods and Neural Networks

We consider the use of Gaussian Processes (GPs) or Neural Networks (NNs) to numerically approximate the solutions to nonlinear partial differential equations (PDEs) with rough forcing or source terms, which commonly arise as pathwise solutions to stochastic PDEs. Kernel methods have recently been generalized to solve nonlinear PDEs by approximating their solutions as the maximum a posteriori estimator of GPs that are conditioned to satisfy the PDE at a finite set of collocation points. The convergence and error guarantees of these methods, however, rely on the PDE being defined in a classical sense and its solution possessing sufficient regularity to belong to the associated reproducing kernel Hilbert space. We propose a generalization of these methods to handle roughly forced nonlinear PDEs while preserving convergence guarantees with an oversmoothing GP kernel that is misspecified relative to the true solution's regularity. This is achieved by conditioning a regular GP to satisfy the PDE with a modified source term in a weak sense (when integrated against a finite number of test functions). This is equivalent to replacing the empirical $L^2$-loss on the PDE constraint by an empirical negative-Sobolev norm. We further show that this loss function can be used to extend physics-informed neural networks (PINNs) to stochastic equations, thereby resulting in a new NN-based variant termed Negative Sobolev Norm-PINN (NeS-PINN).

math.NA↗

arXiv2024

A Mini-Batch Method for Solving Nonlinear PDEs with Gaussian Processes

Gaussian processes (GPs) based methods for solving partial differential equations (PDEs) demonstrate great promise by bridging the gap between the theoretical rigor of traditional numerical algorithms and the flexible design of machine learning solvers. The main bottleneck of GP methods lies in the inversion of a covariance matrix, whose cost grows cubically concerning the size of samples. Drawing inspiration from neural networks, we propose a mini-batch algorithm combined with GPs to solve nonlinear PDEs. A naive deployment of a stochastic gradient descent method for solving PDEs with GPs is challenging, as the objective function in the requisite minimization problem cannot be depicted as the expectation of a finite-dimensional random function. To address this issue, we employ a mini-batch method to the corresponding infinite-dimensional minimization problem over function spaces. The algorithm takes a mini-batch of samples at each step to update the GP model. Thus, the computational cost is allotted to each iteration. Using stability analysis and convexity arguments, we show that the mini-batch method steadily reduces a natural measure of errors towards zero at the rate of $O(1/K+1/M)$, where $K$ is the number of iterations and $M$ is the batch size.

math.NA↗

arXiv2023

Decoding Mean Field Games from Population and Environment Observations By Gaussian Processes

This paper presents a Gaussian Process (GP) framework, a non-parametric technique widely acknowledged for regression and classification tasks, to address inverse problems in mean field games (MFGs). By leveraging GPs, we aim to recover agents' strategic actions and the environment's configurations from partial and noisy observations of the population of agents and the setup of the environment. Our method is a probabilistic tool to infer the behaviors of agents in MFGs from data in scenarios where the comprehensive dataset is either inaccessible or contaminated by noises.

cs.GT↗

arXiv2023

Sparse Gaussian processes for solving nonlinear PDEs

This article proposes an efficient numerical method for solving nonlinear partial differential equations (PDEs) based on sparse Gaussian processes (SGPs). Gaussian processes (GPs) have been extensively studied for solving PDEs by formulating the problem of finding a reproducing kernel Hilbert space (RKHS) to approximate a PDE solution. The approximated solution lies in the span of base functions generated by evaluating derivatives of different orders of kernels at sample points. However, the RKHS specified by GPs can result in an expensive computational burden due to the cubic computation order of the matrix inverse. Therefore, we conjecture that a solution exists on a ``condensed" subspace that can achieve similar approximation performance, and we propose a SGP-based method to reformulate the optimization problem in the ``condensed" subspace. This significantly reduces the computation burden while retaining desirable accuracy. The paper rigorously formulates this problem and provides error analysis and numerical experiments to demonstrate the effectiveness of this method. The numerical experiments show that the SGP method uses fewer than half the uniform samples as inducing points and achieves comparable accuracy to the GP method using the same number of uniform samples, resulting in a significant reduction in computational cost. Our contributions include formulating the nonlinear PDE problem as an optimization problem on a ``condensed" subspace of RKHS using SGP, as well as providing an existence proof and rigorous error analysis. Furthermore, our method can be viewed as an extension of the GP method to account for general positive semi-definite kernels.

math.NA↗

arXiv2023

Forward-backward algorithm for functions with locally Lipschitz gradient: applications to mean field games

In this paper, we provide a generalization of the forward-backward splitting algorithm for minimizing the sum of a proper convex lower semicontinuous function and a differentiable convex function whose gradient satisfies a locally Lipschitztype condition. We prove the convergence of our method and derive a linear convergence rate when the differentiable function is locally strongly convex. We recover classical results in the case when the gradient of the differentiable function is globally Lipschitz continuous and an already known linear convergence rate when the function is globally strongly convex. We apply the algorithm to approximate equilibria of variational mean field game systems with local couplings. Compared with some benchmark algorithms to solve these problems, our numerical tests show similar performances in terms of the number of iterations but an important gain in the required computational time.

math.OC↗

arXiv2022

Numerical methods for Mean field Games based on Gaussian Processes and Fourier Features

In this article, we propose two numerical methods, the Gaussian Process (GP) method and the Fourier Features (FF) algorithm, to solve mean field games (MFGs). The GP algorithm approximates the solution of a MFG with maximum a posteriori probability estimators of GPs conditioned on the partial differential equation (PDE) system of the MFG at a finite number of sample points. The main bottleneck of the GP method is to compute the inverse of a square gram matrix, whose size is proportional to the number of sample points. To improve the performance, we introduce the FF method, whose insight comes from the recent trend of approximating positive definite kernels with random Fourier features. The FF algorithm seeks approximated solutions in the space generated by sampled Fourier features. In the FF method, the size of the matrix to be inverted depends only on the number of Fourier features selected, which is much less than the size of sample points. Hence, the FF method reduces the precomputation time, saves the memory, and achieves comparable accuracy to the GP method. We give the existence and the convergence proofs for both algorithms. The convergence argument of the GP method does not depend on any monotonicity condition, which suggests the potential applications of the GP method to solve MFGs with non-monotone couplings in future work. We show the efficacy of our algorithms through experiments on a stationary MFG with a non-local coupling and on a time-dependent planning problem. We believe that the FF method can also serve as an alternative algorithm to solve general PDEs.

math.NA↗

arXiv2020

The Hessian Riemannian flow and Newton's method for Effective Hamiltonians and Mather measures

Effective Hamiltonians arise in several problems, including homogenization of Hamilton--Jacobi equations, nonlinear control systems, Hamiltonian dynamics, and Aubry--Mather theory. In Aubry--Mather theory, related objects, Mather measures, are also of great importance. Here, we combine ideas from mean-field games with the Hessian Riemannian flow to compute effective Hamiltonians and Mather measures simultaneously. We prove the convergence of the Hessian Riemannian flow in the continuous setting. For the discrete case, we give both the existence and the convergence of the Hessian Riemannian flow. In addition, we explore a variant of Newton's method that greatly improves the performance of the Hessian Riemannian flow. In our numerical experiments, we see that our algorithms preserve the non-negativity of Mather measures and are more stable than {related} methods in problems that are close to singular. Furthermore, our method also provides a way to approximate stationary MFGs.

math.NA↗

arXiv2019

Two-scale homogenization of a stationary mean-field game

In this paper, we characterize the asymptotic behavior of a first-order stationary mean-field game (MFG) with a logarithm coupling, a quadratic Hamiltonian, and a periodically oscillating potential. This study falls into the realm of the homogenization theory, and our main tool is the two-scale convergence. Using this convergence, we rigorously derive the two-scale homogenized and the homogenized MFG problems, which encode the so-called macroscopic or effective behavior of the original oscillating MFG. Moreover, we prove existence and uniqueness of the solution to these limit problems.

math.AP↗

arXiv2015

Existence of positive solutions for an approximation of stationary mean-field games

Here, we consider a regularized mean-field game model that features a low-order regularization. We prove the existence of solutions with positive density. To do so, we combine a priori estimates with the continuation method. In contrast with high-order regularizations, the low-order regularizations are easier to implement numerically. Moreover, our methods give a theoretical foundation for this approach.

math.AP↗