arXiv subjects

Yulong Lu

Publications and source records attributed to Yulong Lu.

At least 19 recordsLinked to original sources

arXiv2026

Feature Learning for the High Dimensional Stationary Sch\"odinger Equation with Deep Ritz Method

This paper investigates feature learning within the framework of the deep Ritz method for solving the stationary Schr\"odinger equation with Neumann boundary conditions. We first analyze the convergence of Riemannian gradient descent in an agnostic setting, where the hypothesis function is restricted to a single-index model while the PDE solution is arbitrary. We prove that gradient descent reaches an approximate global minimum: after T = O(log(1/\epsilon)) iterations, the loss is within \epsilon of a constant multiple of the optimal loss. We then examine the loss landscape when the source term of the PDE itself follows a single-index model, considering hypothesis functions given by either a single-index model or a two-neuron multi-index model. In the single-index case, we show that the minimum Ritz energy is attained at the feature vector aligned with that of the source term. In the two-neuron case, we study the landscape of regularized Ritz losses and characterize how a second feature emerges, given that the first feature is aligned with the source, as the regularization parameter varies. Finally, numerical experiments are presented to validate the feature emergence theory in the two-neuron setting.

math.OC↗

arXiv2026

Posterior Concentration of Bayesian Physics-Informed Neural Networks for Elliptic PDEs

We study the posterior contraction rate of Bayesian Physics-Informed Neural Networks (PINNs) for solving a general class of elliptic partial differential equations (PDEs). We focus on learning of the elliptic equation with a non-homogeneous Dirichlet boundary condition from independent and noisy measurements collected both inside the domain and on the boundary. Assuming that the PDE admits a strong solution in a H\"older space and using with a suitably constructed prior on the neural network weights, we prove that the posterior distribution concentrates around the exact solution at a near-minimax rate. Furthermore, the chosen prior is rate-adaptive: the posterior contracts at an (almost) optimal rate without prior knowledge of the smoothness level of the exact solution. Our results provide statistical guarantees for uncertainty quantification of PDEs via Bayesian PINNs.

math.ST↗

arXiv2026

A Theory of Diversity for Random Matrices with Applications to In-Context Learning of Schr\"odinger Equations

We address the following question: given a collection $\{\mathbf{A}^{(1)}, \dots, \mathbf{A}^{(N)}\}$ of independent $d \times d$ random matrices drawn from a common distribution $\mathbb{P}$, what is the probability that the centralizer of $\{\mathbf{A}^{(1)}, \dots, \mathbf{A}^{(N)}\}$ is trivial? We provide lower bounds on this probability in terms of the sample size $N$ and the dimension $d$ for several families of random matrices which arise from the discretization of linear Schr\"odinger operators with random potentials. When combined with recent work on machine learning theory, our results provide guarantees on the generalization ability of transformer-based neural networks for in-context learning of Schr\"odinger equations.

stat.ML↗

arXiv2026

In-Context Operator Learning on the Space of Probability Measures

We introduce \emph{in-context operator learning on probability measure spaces} for optimal transport (OT). The goal is to learn a single solution operator that maps a pair of distributions to the OT map, using only few-shot samples from each distribution as a prompt and \emph{without} gradient updates at inference. We parameterize the solution operator and develop scaling-law theory in two regimes. In the \emph{nonparametric} setting, when tasks concentrate on a low-intrinsic-dimension manifold of source--target pairs, we establish generalization bounds that quantify how in-context accuracy scales with prompt size, intrinsic task dimension, and model capacity. In the \emph{parametric} setting (e.g., Gaussian families), we give an explicit architecture that recovers the exact OT map in context and provide finite-sample excess-risk bounds. Our numerical experiments on synthetic transports and generative-modeling benchmarks validate the framework.

cs.LG↗

arXiv2025

On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functions

Deep neural networks have been widely used as universal approximators for functions with inherent physical structures, including permutation symmetry. In this paper, we construct symmetric deep neural networks to approximate symmetric Korobov functions and prove that both the convergence rate and the constant prefactor scale at most polynomially with respect to the ambient dimension. This represents a substantial improvement over prior approximation guarantees that suffer from the curse of dimensionality. Building on these approximation bounds, we further derive a generalization-error rate for learning symmetric Korobov functions whose leading factors likewise avoid the curse of dimensionality.

cs.LG↗

arXiv2025

In-Context Learning of Linear Systems: Generalization Theory and Applications to Operator Learning

We study theoretical guarantees for solving linear systems in-context using a linear transformer architecture. For in-domain generalization, we provide neural scaling laws that bound the generalization error in terms of the number of tasks and sizes of samples used in training and inference. For out-of-domain generalization, we find that the behavior of trained transformers under task distribution shifts depends crucially on the distribution of the tasks seen during training. We introduce a novel notion of task diversity and show that it defines a necessary and sufficient condition for pre-trained transformers generalize under task distribution shifts. We also explore applications of learning linear systems in-context, such as to in-context operator learning for PDEs. Finally, we provide some numerical experiments to validate the established theory.

cs.LG↗

arXiv2025

Improving Data Fidelity via Diffusion Model-based Correction and Super-Resolution

We propose a unified diffusion model-based correction and super-resolution method to enhance the fidelity and resolution of diverse low-quality data through a two-step pipeline. First, the correction step employs a novel enhanced stochastic differential editing technique based on an imbalanced perturbation and denoising process, ensuring robust and effective bias correction at the low-resolution level. The robustness and effectiveness of this approach are validated theoretically and experimentally. Next, the super-resolution step leverages cascaded conditional diffusion models to iteratively refine the corrected data to high-resolution. Numerical experiments on three PDE problems and a climate dataset demonstrate that the proposed method effectively enhances low-fidelity, low-resolution data by correcting numerical errors and noise while simultaneously improving resolution to recover fine-scale structures.

math.NA↗

arXiv2025

Convergence of Time-Averaged Mean Field Gradient Descent Dynamics for Continuous Multi-Player Zero-Sum Games

The approximation of mixed Nash equilibria (MNE) for zero-sum games with mean-field interacting players has recently raised much interest in machine learning. In this paper we propose a mean-field gradient descent dynamics for finding the MNE of zero-sum games involving $K$ players with $K\geq 2$. The evolution of the players' strategy distributions follows coupled mean-field gradient descent flows with momentum, incorporating an exponentially discounted time-averaging of gradients. First, in the case of a fixed entropic regularization, we prove an exponential convergence rate for the mean-field dynamics to the mixed Nash equilibrium with respect to the total variation metric. This improves a previous polynomial convergence rate for a similar time-averaged dynamics with different averaging factors. Moreover, unlike previous two-scale approaches for finding the MNE, our approach treats all player types on the same time scale. We also show that with a suitable choice of decreasing temperature, a simulated annealing version of the mean-field dynamics converges to an MNE of the initial unregularized problem.

math.OC↗

arXiv2025

Diffusion-based Models for Unpaired Super-resolution in Fluid Dynamics

High-fidelity, high-resolution numerical simulations are crucial for studying complex multiscale phenomena in fluid dynamics, such as turbulent flows and ocean waves. However, direct numerical simulations with high-resolution solvers are computationally prohibitive. As an alternative, super-resolution techniques enable the enhancement of low-fidelity, low-resolution simulations. However, traditional super-resolution approaches rely on paired low-fidelity, low-resolution and high-fidelity, high-resolution datasets for training, which are often impossible to acquire in complex flow systems. To address this challenge, we propose a novel two-step approach that eliminates the need for paired datasets. First, we perform unpaired domain translation at the low-resolution level using an Enhanced Denoising Diffusion Implicit Bridge. This process transforms low-fidelity, low-resolution inputs into high-fidelity, low-resolution outputs, and we provide a theoretical analysis to highlight the advantages of this enhanced diffusion-based approach. Second, we employ the cascaded Super-Resolution via Repeated Refinement model to upscale the high-fidelity, low-resolution prediction to the high-resolution result. We demonstrate the effectiveness of our approach across three fluid dynamics problems. Moreover, by incorporating a neural operator to learn system dynamics, our method can be extended to improve evolutionary simulations of low-fidelity, low-resolution data.

math.NA↗

arXiv2025

In-Context Learning of Linear Dynamical Systems with Transformers: Approximation Bounds and Depth-Separation

This paper investigates approximation-theoretic aspects of the in-context learning capability of the transformers in representing a family of noisy linear dynamical systems. Our first theoretical result establishes an upper bound on the approximation error of multi-layer transformers with respect to an $L^2$-testing loss uniformly defined across tasks. This result demonstrates that transformers with logarithmic depth can achieve error bounds comparable with those of the least-squares estimator. In contrast, our second result establishes a non-diminishing lower bound on the approximation error for a class of single-layer linear transformers, which suggests a depth-separation phenomenon for transformers in the in-context learning of dynamical systems. Moreover, this second result uncovers a critical distinction in the approximation power of single-layer linear transformers when learning from IID versus non-IID data.

cs.LG↗

arXiv2024

Wall laws for viscous flows in 3D randomly rough pipes: optimal convergence rates and stochastic integrability

This paper is concerned with effective approximations and wall laws of viscous laminar flows in 3D pipes with randomly rough boundaries. The random roughness is characterized by the boundary oscillation scale $\varepsilon \ll 1 $ and a probability space with ergodicity quantified by functional inequalities. The results in this paper generalize the previous work for 2D channel flows with random Lipschitz boundaries to 3D pipe flows with random boundaries of John type. Moreover, we establish the optimal convergence rates and substantially improve the stochastic integrability obtained in the previous studies. Additionally, we prove a refined version of the Poiseuille's law in 3D pipes with random boundaries, which seems unaddressed in the literature. Our systematic approach combines several classical and recent ideas (particularly from homogenization theory), including the Saint-Venant's principle for pipe flows, the large-scale regularity theory over rough boundaries, and applications of functional inequalities.

math.AP↗

arXiv2024

Fully discretized Sobolev gradient flow for the Gross-Pitaevskii eigenvalue problem

This paper studies the numerical approximation of the ground state of the Gross-Pitaevskii (GP) eigenvalue problem with a fully discretized Sobolev gradient flow induced by the $H^1$ norm. For the spatial discretization, we consider the finite element method with quadrature using $P^k$ basis on a simplicial mesh and $Q^k$ basis on a rectangular mesh. We prove the global convergence to a critical point of the discrete GP energy, and establish a local exponential convergence to the ground state under the assumption that the linearized discrete Schrödinger operator has a positive spectral gap. We also show that for the $P^1$ finite element discretization with quadrature on an unstructured shape regular simplicial mesh, the eigengap satisfies a mesh-independent lower bound, which implies a mesh-independent local convergence rate for the proposed discrete gradient flow. Numerical experiments with discretization by high order $Q^k$ spectral element methods in two and three dimensions are provided to validate the efficiency of the proposed method.

math.NA↗

arXiv2024

Generative downscaling of PDE solvers with physics-guided diffusion models

Solving partial differential equations (PDEs) on fine spatio-temporal scales for high-fidelity solutions is critical for numerous scientific breakthroughs. Yet, this process can be prohibitively expensive, owing to the inherent complexities of the problems, including nonlinearity and multiscale phenomena. To speed up large-scale computations, a process known as downscaling is employed, which generates high-fidelity approximate solutions from their low-fidelity counterparts. In this paper, we propose a novel Physics-Guided Diffusion Model (PGDM) for downscaling. Our model, initially trained on a dataset comprising low-and-high-fidelity paired solutions across coarse and fine scales, generates new high-fidelity approximations from any new low-fidelity inputs. These outputs are subsequently refined through fine-tuning, aimed at minimizing the physical discrepancies as defined by the discretized PDEs at the finer scale. We evaluate and benchmark our model's performance against other downscaling baselines in three categories of nonlinear PDEs. Our numerical experiments demonstrate that our model not only outperforms the baselines but also achieves a computational acceleration exceeding tenfold, while maintaining the same level of accuracy as the conventional fine-scale solvers.

math.NA↗

arXiv2024

Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions

While score-based generative models (SGMs) have achieved remarkable success in enormous image generation tasks, their mathematical foundations are still limited. In this paper, we analyze the approximation and generalization of SGMs in learning a family of sub-Gaussian probability distributions. We introduce a notion of complexity for probability distributions in terms of their relative density with respect to the standard Gaussian measure. We prove that if the log-relative density can be locally approximated by a neural network whose parameters can be suitably bounded, then the distribution generated by empirical score matching approximates the target distribution in total variation with a dimension-independent rate. We illustrate our theory through examples, which include certain mixtures of Gaussians. An essential ingredient of our proof is to derive a dimension-free deep neural network approximation rate for the true score function associated with the forward process, which is interesting in its own right.

stat.ML↗

arXiv2023

On the convergence of Sobolev gradient flow for the Gross-Pitaevskii eigenvalue problem

We study the convergences of three projected Sobolev gradient flows to the ground state of the Gross-Pitaevskii eigenvalue problem. They are constructed as the gradient flows of the Gross-Pitaevskii energy functional with respect to the $H^1_0$-metric and two other equivalent metrics on $H_0^1$, including the iterate-independent $a_0$-metric and the iterate-dependent $a_u$-metric. We first prove the energy dissipation property and the global convergence to a critical point of the Gross-Pitaevskii energy for the discrete-time $H^1$ and $a_0$-gradient flow. We also prove local exponential convergence of all three schemes to the ground state.

math.NA↗

arXiv2023

Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Norms

This paper establishes the nearly optimal rate of approximation for deep neural networks (DNNs) when applied to Korobov functions, effectively overcoming the curse of dimensionality. The approximation results presented in this paper are measured with respect to $L_p$ norms and $H^1$ norms. Our achieved approximation rate demonstrates a remarkable "super-convergence" rate, outperforming traditional methods and any continuous function approximator. These results are non-asymptotic, providing error bounds that consider both the width and depth of the networks simultaneously.

math.NA↗

arXiv2023

Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics

Motivated by the challenge of sampling Gibbs measures with nonconvex potentials, we study a continuum birth-death dynamics. We improve results in previous works [51,57] and provide weaker hypotheses under which the probability density of the birth-death governed by Kullback-Leibler divergence or by $χ^2$ divergence converge exponentially fast to the Gibbs equilibrium measure, with a universal rate that is independent of the potential barrier. To build a practical numerical sampler based on the pure birth-death dynamics, we consider an interacting particle system, which is inspired by the gradient flow structure and the classical Fokker-Planck equation and relies on kernel-based approximations of the measure. Using the technique of $Γ$-convergence of gradient flows, we show that on the torus, smooth and bounded positive solutions of the kernelized dynamics converge on finite time intervals, to the pure birth-death dynamics as the kernel bandwidth shrinks to zero. Moreover we provide quantitative estimates on the bias of minimizers of the energy corresponding to the kernelized dynamics. Finally we prove the long-time asymptotic results on the convergence of the asymptotic states of the kernelized dynamics towards the Gibbs measure.

math.AP↗

arXiv2023

Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspective

Finding the mixed Nash equilibria (MNE) of a two-player zero sum continuous game is an important and challenging problem in machine learning. A canonical algorithm to finding the MNE is the noisy gradient descent ascent method which in the infinite particle limit gives rise to the {\em Mean-Field Gradient Descent Ascent} (GDA) dynamics on the space of probability measures. In this paper, we first study the convergence of a two-scale Mean-Field GDA dynamics for finding the MNE of the entropy-regularized objective. More precisely we show that for each finite temperature (or regularization parameter), the two-scale Mean-Field GDA with a suitable {\em finite} scale ratio converges exponentially to the unique MNE without assuming the convexity or concavity of the interaction potential. The key ingredient of our proof lies in the construction of new Lyapunov functions that dissipate exponentially along the Mean-Field GDA. We further study the simulated annealing of the Mean-Field GDA dynamics. We show that with a temperature schedule that decays logarithmically in time the annealed Mean-Field GDA converges to the MNE of the original unregularized objective.

math.OC↗