arXiv subjects

Jiaoyang Huang

Publications and source records attributed to Jiaoyang Huang.

At least 55 records · Page 3Linked to original sources

arXiv2023

Asymptotics of Generalized Bessel Functions and Weight Multiplicities via Large Deviations of Radial Dunkl Processes

This paper studies the asymptotic behavior of several central objects in Dunkl theory as the dimension of the underlying space grows large. Our starting point is the observation that a recent result from the random matrix theory literature implies a large deviations principle for the hydrodynamic limit of radial Dunkl processes. Using this fact, we prove a variational formula for the large-$N$ asymptotics of generalized Bessel functions, as well as a large deviations principle for the more general family of radial Heckman-Opdam processes. As an application, we prove a theorem on the asymptotic behavior of weight multiplicities of irreducible representations of compact or complex simple Lie algebras in the limit of large rank. The theorems in this paper generalize several known results describing analogous asymptotics for Dyson Brownian motion, spherical matrix integrals, and Kostka numbers.

math.PR↗

arXiv2023

Edge Universality of Sparse Random Matrices

We consider the statistics of the extreme eigenvalues of sparse random matrices, a class of random matrices that includes the normalized adjacency matrices of the Erd{\H o}s-R{é}nyi graph $G(N,p)$. Recently, it was shown by Lee, up to an explicit random shift, the optimal rigidity of extreme eigenvalues holds, provided the averaged degree grows with the size of the graph, $pN>N^\varepsilon$. We prove in the same regime, (i) Optimal rigidity holds for all eigenvalues with respect to an explicit random measure. (ii) Up to an explicit random shift, the fluctuations of the extreme eigenvalues are given the Tracy-Widom distribution.

math.PR↗

arXiv2023

Edge Statistics for Lozenge Tilings of Polygons, I: Concentration of Height Function on Strip Domains

In this paper we study uniformly random lozenge tilings of strip domains. Under the assumption that the limiting arctic boundary has at most one cusp, we prove a nearly optimal concentration estimate for the tiling height functions and arctic boundaries on such domains: with overwhelming probability the tiling height function is within $n^δ$ of its limit shape, and the tiling arctic boundary is within $n^{1/3+δ}$ to its limit shape, for arbitrarily small $δ>0$. This concentration result will be used in [AH21] to prove that the edge statistics of simply-connected polygonal domains, subject to a technical assumption on their limit shape, converge to the Airy line ensemble.

math.PR↗

arXiv2023

Large Deviation Principles via Spherical Integrals

In this article, we develop a framework to study the large deviation principle for matrix models and their quantized versions, by tilting the measures using the limits of spherical integrals obtained in [46,47]. As examples, we obtain 1. a large deviation principle for the empirical distribution of the diagonal entries of $UB_NU^*$, for a sequence of $N\times N$ diagonal matrices $B_N$ and unitary Haar distributed matrices $U$; 2. a large deviation upper bound for the empirical eigenvalue distribution of $A_N+UB_NU^*$, for two sequences of $N\times N$ diagonal matrices $A_N, B_N$, and their complementary lower bounds at measures which are described by the free product with amalgamation; 3. a large deviation principle for the Kostka number $K_{\boldsymbolλ_N \boldsymbolη_N}$, for two sequences of partitions $\boldsymbolλ_N, \boldsymbolη_N$ with at most $N$ rows; 4. a large deviation upper bound for the Littlewood-Richardson coefficients $c_{\boldsymbolλ_N \boldsymbol η_N}^{\boldsymbol κ_N}$, for three sequences of partitions $\boldsymbolλ_N, \boldsymbol η_N, \boldsymbol κ_N$ with at most $N$ rows, and their complementary lower bounds at nice measures.

math.PR↗

arXiv2023

PatchGT: Transformer over Non-trainable Clusters for Learning Graph Representations

Recently the Transformer structure has shown good performances in graph learning tasks. However, these Transformer models directly work on graph nodes and may have difficulties learning high-level information. Inspired by the vision transformer, which applies to image patches, we propose a new Transformer-based graph neural network: Patch Graph Transformer (PatchGT). Unlike previous transformer-based models for learning graph representations, PatchGT learns from non-trainable graph patches, not from nodes directly. It can help save computation and improve the model performance. The key idea is to segment a graph into patches based on spectral clustering without any trainable parameters, with which the model can first use GNN layers to learn patch-level representations and then use Transformer to obtain graph-level representations. The architecture leverages the spectral information of graphs and combines the strengths of GNNs and Transformers. Further, we show the limitations of previous hierarchical trainable clusters theoretically and empirically. We also prove the proposed non-trainable spectral clustering method is permutation invariant and can help address the information bottlenecks in the graph. PatchGT achieves higher expressiveness than 1-WL-type GNNs, and the empirical study shows that PatchGT achieves competitive performances on benchmark datasets and provides interpretability to its predictions. The implementation of our algorithm is released at our Github repo: https://github.com/tufts-ml/PatchGT.

cs.LG↗

arXiv2022

Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems

We consider Bayesian inference for large scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov chain Monte Carlo approaches infeasible, since they typically require $O(10^4)$ model runs, or more. Moreover, the forward model is often given as a black box or is impractical to differentiate. Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems. The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system into which the inverse problem is embedded as an observation operator. Theoretical properties of the mean-field model are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it. This suggests that, for nonlinear problems which are close to Gaussian, sequentially computing this law provides the basis for efficient iterative methods to approximate the Bayesian posterior. Ensemble methods are applied to obtain interacting particle system approximations of the filtering distribution of the mean-field model; and practical strategies to further reduce the computational and memory cost of the methodology are presented, including low-rank approximation and a bi-fidelity approach. The effectiveness of the framework is demonstrated in several numerical experiments, including proof-of-concept linear/nonlinear examples and two large-scale applications: learning of permeability parameters in subsurface flow; and learning subgrid-scale parameters in a global climate model from time-averaged statistics.

math.NA↗

arXiv2022

Robustness Implies Generalization via Data-Dependent Generalization Bounds

This paper proves that robustness implies generalization via data-dependent generalization bounds. As a result, robustness and generalization are shown to be connected closely in a data-dependent manner. Our bounds improve previous bounds in two directions, to solve an open problem that has seen little development since 2010. The first is to reduce the dependence on the covering number. The second is to remove the dependence on the hypothesis space. We present several examples, including ones for lasso and deep learning, in which our bounds are provably preferable. The experiments on real-world data and theoretical models demonstrate near-exponential improvements in various situations. To achieve these improvements, we do not require additional assumptions on the unknown distribution; instead, we only incorporate an observable and computable property of the training samples. A key technical innovation is an improved concentration bound for multinomial random variables that is of independent interest beyond robustness and generalization.

cs.LG↗

arXiv2021

Long Random Matrices and Tensor Unfolding

In this paper, we consider the singular values and singular vectors of low rank perturbations of large rectangular random matrices, in the regime the matrix is "long": we allow the number of rows (columns) to grow polynomially in the number of columns (rows). We prove there exists a critical signal-to-noise ratio (depending on the dimensions of the matrix), and the extreme singular values and singular vectors exhibit a BBP type phase transition. As a main application, we investigate the tensor unfolding algorithm for the asymmetric rank-one spiked tensor model, and obtain an exact threshold, which is independent of the procedure of tensor unfolding. If the signal-to-noise ratio is above the threshold, tensor unfolding detects the signals; otherwise, it fails to capture the signals.

math.PR↗

arXiv2021

Large Deviations Asymptotics of Rectangular Spherical Integral

In this article we study the Dyson Bessel process, which describes the evolution of singular values of rectangular matrix Brownian motions, and prove a large deviation principle for its empirical particle density. We then use it to obtain the asymptotics of the so-called rectangular spherical integrals as $m,n$ go to infinity while $m/n$ converges.

math.PR↗

arXiv2021

Unscented Kalman Inversion: Efficient Gaussian Approximation to the Posterior Distribution

The unscented Kalman inversion (UKI) method presented in [1] is a general derivative-free approach for the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategies, convergence property, and speed-up strategies [1,2] of the UKI are thoroughly studied, and the method is capable of handling noisy observation data and solving chaotic inverse problems. In this paper, we study the uncertainty quantification capability of the UKI. We propose a modified UKI, which allows to well approximate the mean and covariance of the posterior distribution for well-posed inverse problems with large observation data. Theoretical guarantees for both linear and nonlinear inverse problems are presented. Numerical results, including learning of permeability parameters in subsurface flow and of the Navier-Stokes initial condition from solution data at positive times are presented. The results obtained by the UKI require only $O(10)$ iterations, and match well with the expected results obtained by the Markov Chain Monte Carlo method.

math.NA↗

arXiv2021

$β$-Nonintersecting Poisson Random Walks: Law of Large Numbers and Central Limit Theorems

We study the $β$ analogue of the nonintersecting Poisson random walks. We derive a stochastic differential equation of the Stieltjes transform of the empirical measure process, which can be viewed as a dynamical version of the Nekrasov's equation in [7, Section 4]. We find that the empirical measure process converges weakly in the space of cádlág measure-valued processes to a deterministic process, characterized by the quantized free convolution, as introduced in [11]. For suitable initial data, we prove that the rescaled empirical measure process converges weakly in the space of distributions acting on analytic test functions to a Gaussian process. The means and the covariances are universal, and coincide with those of $β$-Dyson Brownian motions with the initial data constructed by the Markov-Krein correspondence. Especially, the covariance structure can be described in terms of the Gaussian Free Field. Our proof relies on integrable features of the generators of the $β$-nonintersecting Poisson random walks, the method of characteristics, and a coupling technique for Poisson random walks.

math.PR↗

arXiv2021

Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model

The unscented Kalman inversion (UKI) presented in [1] is a general derivative-free approach to solving the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategy and convergence property of the UKI are thoroughly studied, and the method is demonstrated effectively handling noisy observation data and solving chaotic inverse problems. In this paper, we aim to make the UKI more efficient in terms of computational and memory costs for large scale inverse problems. We take advantages of the low-rank covariance structure to reduce the number of forward problem evaluations and the memory cost, related to the need to propagate large covariance matrices. And we leverage reduced-order model techniques to further speed up these forward evaluations. The effectiveness of the enhanced UKI is demonstrated on a barotropic model inverse problem with O($10^5$) unknown parameters and a 3D generalized circulation model (GCM) inverse problem, where each iteration is as efficient as that of gradient-based optimization methods.

math.NA↗

arXiv2020

Power Iteration for Tensor PCA

In this paper, we study the power iteration algorithm for the spiked tensor model, as introduced in [44]. We give necessary and sufficient conditions for the convergence of the power iteration algorithm. When the power iteration algorithm converges, for the rank one spiked tensor model, we show the estimators for the spike strength and linear functionals of the signal are asymptotically Gaussian; for the multi-rank spiked tensor model, we show the estimators are asymptotically mixtures of Gaussian. This new phenomenon is different from the spiked matrix model. Using these asymptotic results of our estimators, we construct valid and efficient confidence intervals for spike strengths and linear functionals of the signals.

math.ST↗

arXiv2020

Height Fluctuations of Random Lozenge Tilings Through Nonintersecting Random Walks

In this paper we study height fluctuations of random lozenge tilings of polygonal domains on the triangular lattice through nonintersecting Bernoulli random walks. For a large class of polygons which have exactly one horizontal upper boundary edge, we show that these random height functions converge to a Gaussian Free Field as predicted by Kenyon and Okounkov [28]. A key ingredient of our proof is a dynamical version of the discrete loop equations as introduced by Borodin, Guionnet and Gorin [5], which might be of independent interest.

math.PR↗

arXiv2020

Edge Universality for Nonintersecting Brownian Bridges

In this paper we study fluctuations of extreme particles of nonintersecting Brownian bridges starting from $a_1\leq a_2\leq \cdots \leq a_n$ at time $t=0$ and ending at $b_1\leq b_2\leq \cdots\leq b_n$ at time $t=1$, where $μ_{A_n}=(1/n)\sum_{i}δ_{a_i}, μ_{B_n}=(1/n)\sum_i δ_{b_i}$ are discretization of probability measures $μ_A, μ_B$. Under regularity assumptions of $μ_A, μ_B$, we show as the number of particles $n$ goes to infinity, fluctuations of extreme particles at any time $0<t<1$, after proper rescaling, are asymptotically universal, converging to the Airy point process.

math.PR↗

arXiv2020

Towards Understanding the Dynamics of the First-Order Adversaries

An acknowledged weakness of neural networks is their vulnerability to adversarial perturbations to the inputs. To improve the robustness of these models, one of the most popular defense mechanisms is to alternatively maximize the loss over the constrained perturbations (or called adversaries) on the inputs using projected gradient ascent and minimize over weights. In this paper, we analyze the dynamics of the maximization step towards understanding the experimentally observed effectiveness of this defense mechanism. Specifically, we investigate the non-concave landscape of the adversaries for a two-layer neural network with a quadratic loss. Our main result proves that projected gradient ascent finds a local maximum of this non-concave problem in a polynomial number of iterations with high probability. To our knowledge, this is the first work that provides a convergence analysis of the first-order adversaries. Moreover, our analysis demonstrates that, in the initial phase of adversarial training, the scale of the inputs matters in the sense that a smaller input scale leads to faster convergence of adversarial training and a "more regular" landscape. Finally, we show that these theoretical findings are in excellent agreement with a series of experiments.

cs.LG↗

arXiv2020

Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizes

In this paper, we theoretically prove that gradient descent can find a global minimum of non-convex optimization of all layers for nonlinear deep neural networks of sizes commonly encountered in practice. The theory developed in this paper only requires the practical degrees of over-parameterization unlike previous theories. Our theory only requires the number of trainable parameters to increase linearly as the number of training samples increases. This allows the size of the deep neural networks to be consistent with practice and to be several orders of magnitude smaller than that required by the previous theories. Moreover, we prove that the linear increase of the size of the network is the optimal rate and that it cannot be improved, except by a logarithmic factor. Furthermore, deep neural networks with the trainability guarantee are shown to generalize well to unseen test samples with a natural dataset but not a random dataset.

stat.ML↗

arXiv2020

Edge rigidity and universality of random regular graphs of intermediate degree

For random $d$-regular graphs on $N$ vertices with $1 \ll d \ll N^{2/3}$, we develop a $d^{-1/2}$ expansion of the local eigenvalue distribution about the Kesten-McKay law up to order $d^{-3}$. This result is valid up to the edge of the spectrum. It implies that the eigenvalues of such random regular graphs are more rigid than those of Erdős-Rényi graphs of the same average degree. As a first application, for $1 \ll d \ll N^{2/3}$, we show that all nontrivial eigenvalues of the adjacency matrix are with very high probability bounded in absolute value by $(2 + o(1)) \sqrt{d - 1}$. As a second application, for $N^{2/9} \ll d \ll N^{1/3}$, we prove that the extremal eigenvalues are concentrated at scale $N^{-2/3}$ and their fluctuations are governed by Tracy-Widom statistics. Thus, in the same regime of $d$, $52\%$ of all $d$-regular graphs have second-largest eigenvalue strictly less than $2 \sqrt{d - 1}$.

math.PR↗