arXiv subjects

Zhengjiang Lin

Publications and source records attributed to Zhengjiang Lin.

13 recordsLinked to original sources

arXiv2026

A Vibrating Clamped Plate Resembles a Vibrating Membrane at High Frequency

We study the clamped plate eigenvalue problem. We show that clamped plate eigenfunctions can be approximated in the $L^2$-sense by their oscillatory components, which solve the Helmholtz equation, with an error that decays exponentially fast in the frequency on any compact subdomain. Additionally, we establish an upper bound on the boundary localization of clamped plate eigenfunctions analogous to the one observed for membrane eigenfunctions.

math.AP↗

arXiv2026

Propagation of Chaos in Contextual Flow Maps

We develop a quantitative statistical theory of transformers in the large-context regime by adopting the abstraction of contextual flow maps (CFMs): dynamical systems that evolve a distinguished token in the presence of a contextual measure across a stack of attention blocks. Within this framework, the finite-context model approximates an idealized infinite-context system in which the contextual measure is replaced by its underlying population, so that the context length $n$ becomes a statistical resource. Exploiting the McKean--Vlasov structure of the dynamics and the classical machinery of propagation of chaos, we establish a forward bound controlling the deviation between the finite- and infinite-context CFMs uniformly along depth, and a backward bound controlling the deviation between the corresponding training trajectories uniformly across iterations of online gradient descent. Both bounds achieve the optimal Wasserstein rate $n^{-1/d}$ for general CFMs and parametric rate $n^{-1/2}$ for a restricted class of CFMs that includes transformers as a special case. The analysis rests on a new Eulerian adjoint formulation of the loss gradient and stability estimates for the resulting forward--adjoint system, both of which may be of independent interest.

cs.LG↗

arXiv2026

Scaling Limits of Long-Context Transformers

We study the long-context limit of softmax self-attention with a fixed query and a random context of $n$ i.i.d. keys on the sphere, viewing the inverse temperature $\beta_n$ as the scaling parameter that decides whether attention degenerates into uniform averaging or collapses onto the single closest key. We show that the critical scale at which selectivity emerges is determined by the local exponent of the distance-to-query distribution near zero rather than by global features of the context, and scales like $\beta_n^\ast \asymp n^{2/(d-1)}$ for uniform keys on $\mathbb{S}^{d-1}$. Furthermore, we characterize the limiting laws of the ordered attention weights and of the attention output across all regimes of $\beta_n$: a subcritical regime in which the output reduces to a local average around $q$ with explicit deterministic bias and Gaussian fluctuations; a critical regime in which a finite collection of nearest keys retains macroscopic mass without single-key collapse; and a supercritical regime in which all mass concentrates on the closest key. Of notable interest is the subcritical case with identity value matrix where the attention map approximately implements a backward heat equation.

cs.LG↗

arXiv2026

Whispering Gallery Modes for Semilinear Dirichlet Eigenvalue Problems

We study the boundary localization phenomenon, known as whispering gallery modes, for weak solutions to semilinear Dirichlet eigenvalue problems in the unit ball $B_1 \subseteq \mathbb{R}^d$ ($d \geq 2$) of the form \[ \begin{cases} -\Delta u + f(u) = \lambda u & \text{in } B_1,\\ u = 0 & \text{on } \partial B_1. \end{cases} \] Here, $f = F'$ where $F$ is a nonnegative $C^2$-function with superquadratic polynomial growth. We prove the existence of a sequence of solutions $(u_n, \lambda_n)$ with $\lambda_n \to +\infty$ such that, for any $\tau \in (0,1)$, \[ \lim_{n \to \infty} \frac{E_\tau(u_n)}{E_1(u_n)} = 0, \] where $E_\rho(u) = \int_{B_\rho} \bigl( \frac{1}{2} |\nabla u|^2 + F(u) \bigr) \, dx$ is the energy over the ball of radius $\rho$. This establishes that the energy of these high-eigenvalue solutions concentrates near the boundary, extending the classical whispering gallery mode phenomenon from linear Laplacian eigenfunctions to the semilinear setting. As a direct application, the case $F(u) = u^4 /4$ yields, after suitable scaling, a sequence of high-eigenvalue boundary-concentrating solutions, providing a nonlinear analogue of whispering gallery modes for the Allen-Cahn equations. The approach combines spectral properties of the linear Laplacian with local nonlinear bifurcation and is expected to adapt to related interior localization phenomena, such as bouncing ball modes, in other geometries where linear eigenfunctions exhibit analogous localization behaviors (e.g., filled ellipses or elliptical annuli).

math.AP↗

arXiv2025

Critical attention scaling in long-context transformers

As large language models scale to longer contexts, attention layers suffer from a fundamental pathology: attention scores collapse toward uniformity as context length $n$ increases, causing tokens to cluster excessively, a phenomenon known as rank-collapse. While $\textit{attention scaling}$ effectively addresses this deficiency by rescaling attention scores with a polylogarithmic factor $\beta_n$, theoretical justification for this approach remains lacking. We analyze a simplified yet tractable model that magnifies the effect of attention scaling. In this model, attention exhibits a phase transition governed by the scaling factor $\beta_n$: insufficient scaling collapses all tokens to a single direction, while excessive scaling reduces attention to identity, thereby eliminating meaningful interactions between tokens. Our main result identifies the critical scaling $\beta_n \asymp \log n$ and provides a rigorous justification for attention scaling in YaRN and Qwen, clarifying why logarithmic scaling maintains sparse, content-adaptive attention at large context lengths.

cs.LG↗

arXiv2025

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models

Diffusion probabilistic models generate samples by learning to reverse a noise-injection process that transforms data into noise. A key development is the reformulation of the reverse sampling process as a deterministic probability flow ordinary differential equation (ODE), which allows for efficient sampling using high-order numerical solvers. Unlike traditional time integrator analysis, the accuracy of this sampling procedure depends not only on numerical integration errors but also on the approximation quality and regularity of the learned score function, as well as their interaction. In this work, we present a rigorous convergence analysis of deterministic samplers derived from probability flow ODEs for general forward processes with arbitrary variance schedules. Specifically, we develop and analyze $p$-th order (exponential) Runge-Kutta schemes, under the practical assumption that the first and second derivatives of the learned score function are bounded. We prove that the total variation distance between the generated and target distributions can be bounded as \begin{align*} O\bigl(d^{\frac{7}{4}}\varepsilon_{\text{score}}^{\frac{1}{2}} +d(dH_{\max})^p\bigr), \end{align*} where $\varepsilon^2_{\text{score}}$ denotes the $L^2$ error in the score function approximation, $d$ is the data dimension, and $H_{\max}$ represents the maximum solver step size. Numerical experiments on benchmark datasets further confirm that the derivatives of the learned score function are bounded in practice.

cs.LG↗

arXiv2025

Quantitative Clustering in Mean-Field Transformer Models

The evolution of tokens through deep transformer models can be modeled as an interacting particle system that has been shown to exhibit an asymptotic clustering behavior akin to the synchronization phenomenon in Kuramoto models. In this work, we investigate the long-time clustering of mean-field transformer models. More precisely, under suitable assumptions on the transformer model parameters, we establish that any suitably regular mean-field initialization synchronizes exponentially fast to a Dirac point mass, with explicit quantitative convergence rates.

cs.LG↗

arXiv2025

Residual connections provably mitigate oversmoothing in graph neural networks

Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.

cs.LG↗

arXiv2024

Convergence Analysis of Probability Flow ODE for Score-based Generative Models

Score-based generative models have emerged as a powerful approach for sampling high-dimensional probability distributions. Despite their effectiveness, their theoretical underpinnings remain relatively underdeveloped. In this work, we study the convergence properties of deterministic samplers based on probability flow ODEs from both theoretical and numerical perspectives. Assuming access to $L^2$-accurate estimates of the score function, we prove the total variation between the target and the generated data distributions can be bounded above by $\mathcal{O}(d^{3/4}\delta^{1/2})$ in the continuous time level, where $d$ denotes the data dimension and $\delta$ represents the $L^2$-score matching error. For practical implementations using a $p$-th order Runge-Kutta integrator with step size $h$, we establish error bounds of $\mathcal{O}(d^{3/4}\delta^{1/2} + d\cdot(dh)^p)$ at the discrete level. Finally, we present numerical studies on problems up to 128 dimensions to verify our theory.

cs.LG↗

arXiv2023

Nonlocal energy functionals and determinantal point processes on non-smooth domains

Given a nonnegative integrable function $J$ on $\mathbb{R}^n$, we relate the asymptotic properties of the nonlocal energy functional \begin{equation*} \int_{\Omega} \int_{\Omega^c} J \bigg(\frac{x-y}{t}\bigg) \ dx dy \end{equation*} as $t \to 0^+$ with the boundary properties of a given domain $\Omega \subset \mathbb{R}^n$. Then, we use these asymptotic properties to study the fluctuations of many determinantal point processes, and show that their variances measure the Minkowski dimension of $\partial \Omega$.

math.AP↗

arXiv2022

On the topology of Gaussian random zero sets

We study the asymptotic laws for the number, Betti numbers, and isotopy classes of connected components of zero sets of real Gaussian random fields, where the random zero sets almost surely consist of submanifolds of codimension greater than or equal to one. Our results include `random knots' as a special case. Our work is closely related to a series of questions posed by Berry in [4,5]; in particular, our results apply to the ensembles of random knots that appear in the complex arithmetic random waves (Example 1.5), the Bargmann-Fock model (Example 1.1), Black-Body radiation (Example 1.2), and Berry's monochromatic random waves. Our proofs combine techniques introduced for level sets of random scalar-valued functions with methods from differential geometry and differential topology.

math.PR↗

arXiv2021

Boundary Harnack Principle on Nodal Domains

We study some geometric and potential theoretic properties of nodal domains of solutions to certain uniformly elliptic equations. In particular, we establish corkscrew conditions, Carleson type estimates and boundary Harnack inequalities on a class of nodal domains.

math.AP↗

arXiv2020

Bifurcation of perturbations of non-generic closed self-shrinkers

We discover a bifurcation of the perturbations of non-generic closed self-shrinkers. If the generic perturbation is outward, then the next mean curvature flow singularity is cylindrical and collapsing from outside; if the generic perturbation is inward, then the next mean curvature flow singularity is cylindrical and collapsing from inside.

math.DG↗