arXiv subjects

Hanbaek Lyu

Publications and source records attributed to Hanbaek Lyu.

At least 19 recordsLinked to original sources

arXiv2026

Finding Koopman Invariant Subspaces via Personalized PageRank

Selecting a finite dictionary of observables whose span is Koopman-invariant is a central challenge in data-driven Koopman operator approximation. We address this problem by exploiting zero-block structure in Extended Dynamic Mode Decomposition (EDMD) matrices. We show that any sub-dictionary whose span is Koopman-invariant induces an exact zero block in the EDMD matrix, even for finite data. We then show that such blocks can be detected by applying PageRank to a row-normalized EDMD matrix constructed from a large initial dictionary. The theory extends to approximately invariant subspaces and yields stronger guarantees for personalized PageRank (PPR) when the seed observables lie inside the target block and reach all observables in that block. Combining EDMD concentration bounds with PageRank perturbation theory gives end-to-end detection guarantees with $O(1/\sqrt{M})$ finite-sample scaling and explicit constants. More generally, without assuming an invariant subspace exists, high PPR mass on a sub-dictionary controls discounted multi-step leakage from the seed observables. Numerical experiments on the Duffing oscillator, Van der Pol oscillator, Lorenz system, and a three-well Ramachandran potential suggest that the method identifies compact, interpretable dictionaries with accurate predictions.

math.DS↗

arXiv2026

Scaling limit of Sinkhorn-rescaled Random Matrices via Stability of Static Schr\"odinger Bridges

We analyze the asymptotic behavior and scaling limits of large random matrices rescaled via the Sinkhorn algorithm to match prescribed row and column margins. For a random matrix with independent sub-exponential entries, we show that its Sinkhorn rescaling concentrates around the rescaling of its mean matrix, both at the level of the Schr\"odinger potentials and as random measures on the unit square, with explicit non-asymptotic rates. As the dimensions grow, the rescaled random matrix converges to the continuous static Schr\"odinger bridge (SSB) determined by the limiting margins and reference density. Around this scaling limit we develop a fluctuation theory: bulk rigidity for the empirical spectral distribution of the associated sample covariance matrix, and a central limit theorem for the empirical Schr\"odinger potentials of the rescaled empirical mean. Our analysis is driven by a new quantitative stability theory for the SSB, developed in three forms: Lipschitz continuity in the Hellinger distance under perturbations of the reference measure (kernel stability); H\"older-$1/2$ continuity in the Hellinger distance under $L^1$ perturbations of the margins (margin stability); and $L^\infty$ stability of the discrete Schr\"odinger potentials under margin perturbation (potential stability). Translated to the discrete random-matrix setting, these bounds yield the concentration and scaling-limit results, while a local law for random Gram matrices with a non-uniform variance profile drives the bulk rigidity. Our SSB stability theory may be of independent interest.

math.PR↗

arXiv2025

Diffusive Scaling limit of stochastic Box-Ball systems and PushTASEP

We introduce the Stochastic Box-Ball System (SBBS), a probabilistic cellular automaton that generalizes the classic Takahashi-Satsuma Box-Ball System. In SBBS, particles are transported by a carrier with a fixed capacity that may fail to pick up any given particle with a fixed probability $\epsilon$. This model interpolates between two known integrable systems: the Box-Ball System (as $\epsilon\rightarrow 0$) and the PushTASEP (as $\epsilon\rightarrow 1$). We show that the long-term behavior of SBBS is governed by isolated particles and the occasional emergence of short solitons, which can form longer solitons but are more likely to fall apart. More precisely, we first show that all particles are isolated except for a $1/\sqrt{n}$-fraction of times in any given $n$ steps, and solitons keep forming for this fraction of times. We then show that under diffusive scaling, both SBBS (for any carrier capacity) and PushTASEP converge weakly to semimartingale reflecting Brownian Motions (SRBMs) on the Weyl chamber with explicit covariance and reflection matrices, which are consistent with the microscale relations between these systems. The reflection matrix for SBBS is determined by how 2-solitons behave and exhibit ``solitonic bias'' visible in the diffusive scale. Our proof relies on a new, extended SRBM invariance principle that we develop in this work. This principle can handle processes with complex boundary behavior that can be written as "overdetermined" Skorokhod decompositions, which is crucial for analyzing the complex solitonic interaction in SBBS. We believe this tool may be of independent interest.

math.PR↗

arXiv2025

Regularized Overestimated Newton

We propose Regularized Overestimated Newton (RON), a Newton-type method with low per-iteration cost and strong global and local convergence guarantees for smooth convex optimization. RON interpolates between gradient descent and globally regularized Newton, with behavior determined by the largest Hessian overestimation error. Globally, when the optimality gap of the objective is large, RON achieves an accelerated $O(n^{-2})$ convergence rate; when small, its rate becomes $O(n^{-1})$. Locally, RON converges superlinearly and linearly when the overestimation is exact and inexact, respectively, toward possibly non-isolated minima under the local Quadratic Growth (QG) condition. The linear rate is governed by an improved effective condition number depending on the overestimation error. Leveraging a recent randomized rank-$k$ Hessian approximation algorithm, we obtain a practical variant with $O(\text{dim}\cdot k^2)$ cost per iteration. When the Hessian rank is uniformly below $k$, RON achieves a per-iteration cost comparable to that of first-order methods while retaining the superior convergence rates even in degenerate local landscapes. We validate our theoretical findings through experiments on entropic optimal transport and inverse problems.

math.OC↗

arXiv2025

Sobolev acceleration for neural networks

Sobolev training, which integrates target derivatives into the loss functions, has been shown to accelerate convergence and improve generalization compared to conventional $L^2$ training. However, the underlying mechanisms of this training method remain only partially understood. In this work, we present the first rigorous theoretical framework proving that Sobolev training accelerates the convergence of Rectified Linear Unit (ReLU) networks. Under a student-teacher framework with Gaussian inputs and shallow architectures, we derive exact formulas for population gradients and Hessians, and quantify the improvements in conditioning of the loss landscape and gradient-flow convergence rates. Extensive numerical experiments validate our theoretical findings and show that the benefits of Sobolev training extend to modern deep learning tasks.

cs.LG↗

arXiv2025

Sample Complexity of Branch-length Estimation by Maximum Likelihood

We consider the branch-length estimation problem on a bifurcating tree: a character evolves along the edges of a binary tree according to a two-state symmetric Markov process, and we seek to recover the edge transition probabilities from repeated observations at the leaves. This problem arises in phylogenetics, and is related to latent tree graphical model inference. In general, the log-likelihood function is non-concave and may admit many critical points. Nevertheless, simple coordinate maximization has been known to perform well in practice, defying the complexity of the likelihood landscape. In this work, we provide the first theoretical guarantee as to why this might be the case. We show that deep inside the Kesten-Stigum reconstruction regime, provided with polynomially many $m$ samples (assuming the tree is balanced), there exists a universal parameter regime (independent of the size of the tree) where the log-likelihood function is strongly concave and smooth with high probability. On this high-probability likelihood landscape event, we show that the standard coordinate maximization algorithm converges exponentially fast to the maximum likelihood estimator, which is within $O(1/\sqrt{m})$ from the true parameter, provided a sufficiently close initial point.

stat.CO↗

arXiv2025

Likelihood landscape of binary latent model on a tree

We investigate the optimization landscape of maximum likelihood estimation (MLE) for the Cavender-Farris-Neyman (CFN) model, a two-state latent tree model fundamental to statistical phylogenetics and the ferromagnetic Ising model. Although the log-likelihood function is non-concave and may admit many critical points, simple coordinate maximization algorithms are remarkably effective in practice. We provide the first theoretical justification for this success. We prove that sufficiently deep inside the reconstruction regime, the population log-likelihood is strongly concave and smooth within a box around the true parameter, whose size is independent of tree topology and number of leaves. This fundamental result implies that the empirical landscape shares these regularity properties with high probability given polynomial sample complexity and also that coordinate maximization converges exponentially fast to an $O(1/\sqrt{m})$-consistent MLE. Our analysis centers on a novel decay property of the population Hessian: diagonal entries remain large while off-diagonal entries decay exponentially with graph distance. These results provide rigorous theoretical evidence for the efficacy of likelihood-based tree inference and suggest broader principles for latent variable models.

math.ST↗

arXiv2025

Likelihood-Based Root State Reconstruction on a Tree: Sensitivity to Parameters and Applications

We consider a broadcasting problem on a tree where a binary digit (e.g., a spin or a nucleotide's purine/pyrimidine type) is propagated from the root to the leaves through symmetric noisy channels on the edges that randomly flip the state with edge-dependent probabilities. The goal of the reconstruction problem is to infer the root state given the observations at the leaves only. Specifically, we study the sensitivity of maximum likelihood estimation (MLE) to uncertainty in the edge parameters under this model, which is also known as the Cavender-Farris-Neyman (CFN) model. Our main result shows that when the true flip probabilities are sufficiently small, the posterior root mean (or magnetization of the root) under estimated parameters (within a constant factor) agrees with the root spin with high probability and deviates significantly from it with negligible probability. This provides theoretical justification for the practical use of MLE in ancestral sequence reconstruction in phylogenetics, where branch lengths (i.e., the edge parameters) must be estimated. As a separate application, we derive an approximation for the gradient of the population log-likelihood of the leaf states under the CFN model, with implications for branch length estimation via coordinate maximization.

math.PR↗

arXiv2024

Large random matrices with given margins

We study large random matrices with i.i.d. entries conditioned to have prescribed row and column sums (margins), a problem connected to relative entropy minimization, Schr\"odinger bridges, contingency tables, and random graphs with given degree sequences. Our central result is a `transference principle': the complex margin-conditioned matrix can be closely approximated by a simpler matrix whose entries are independent and drawn from an exponential tilting of the original model. The tilt parameters are determined by the sum of two potentials. We establish phase diagrams for `tame margins', where these potentials are uniformly bounded. This framework resolves a 2011 conjecture by Chatterjee, Diaconis, and Sly on $\delta$-tame degree sequences and generalizes a sharp phase transition in contingency tables obtained by Dittmer, Lyu, and Pak in 2020. For tame margins, we show that a generalized Sinkhorn algorithm can compute the potentials at a dimension-free exponential rate. Our limit theory further establishes that for a convergent sequence of tame margins, the potentials converge as fast as the margins converge. We apply this framework and obtain several key results for the conditioned matrix: The marginal distribution of any single entry is asymptotically an exponential tilting of the base measure, resolving a 2010 conjecture by Barvinok on contingency tables. The conditioned matrix concentrates in cut norm around a `typical table' (the expectation of the tilted model), which acts as a static Schr\"odinger bridge between the margins. The empirical singular value distribution of the rescaled matrix converges to an explicit law determined by the variance profile of the tilted model. In particular, we confirm the universality of the Marchenko-Pastur law for constant linear margins.

math.PR↗

arXiv2024

Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms

We analyze inexact Riemannian gradient descent (RGD) where Riemannian gradients and retractions are inexactly (and cheaply) computed. Our focus is on understanding when inexact RGD converges and what is the complexity in the general nonconvex and constrained setting. We answer these questions in a general framework of tangential Block Majorization-Minimization (tBMM). We establish that tBMM converges to an $\epsilon$-stationary point within $O(\epsilon^{-2})$ iterations. Under a mild assumption, the results still hold when the subproblem is solved inexactly in each iteration provided the total optimality gap is bounded. Our general analysis applies to a wide range of classical algorithms with Riemannian constraints including inexact RGD and proximal gradient method on Stiefel manifolds. We numerically validate that tBMM shows improved performance over existing methods when applied to various problems, including nonnegative tensor decomposition with Riemannian constraints, regularized nonnegative matrix factorization, and low-rank matrix recovery problems.

math.OC↗

arXiv2024

New directions in algebraic statistics: Three challenges from 2023

In the last quarter of a century, algebraic statistics has established itself as an expanding field which uses multilinear algebra, commutative algebra, computational algebra, geometry, and combinatorics to tackle problems in mathematical statistics. These developments have found applications in a growing number of areas, including biology, neuroscience, economics, and social sciences. Naturally, new connections continue to be made with other areas of mathematics and statistics. This paper outlines three such connections: to statistical models used in educational testing, to a classification problem for a family of nonparametric regression models, and to phase transition phenomena under uniform sampling of contingency tables. We illustrate the motivating problems, each of which is for algebraic statistics a new direction, and demonstrate an enhancement of related methodologies.

math.ST↗

arXiv2024

On the Complexity of First-Order Methods in Stochastic Bilevel Optimization

We consider the problem of finding stationary points in Bilevel optimization when the lower-level problem is unconstrained and strongly convex. The problem has been extensively studied in recent years; the main technical challenge is to keep track of lower-level solutions $y^*(x)$ in response to the changes in the upper-level variables $x$. Subsequently, all existing approaches tie their analyses to a genie algorithm that knows lower-level solutions and, therefore, need not query any points far from them. We consider a dual question to such approaches: suppose we have an oracle, which we call $y^*$-aware, that returns an $O(\epsilon)$-estimate of the lower-level solution, in addition to first-order gradient estimators {\it locally unbiased} within the $\Theta(\epsilon)$-ball around $y^*(x)$. We study the complexity of finding stationary points with such an $y^*$-aware oracle: we propose a simple first-order method that converges to an $\epsilon$ stationary point using $O(\epsilon^{-6}), O(\epsilon^{-4})$ access to first-order $y^*$-aware oracles. Our upper bounds also apply to standard unbiased first-order oracles, improving the best-known complexity of first-order methods by $O(\epsilon)$ with minimal assumptions. We then provide the matching $\Omega(\epsilon^{-6})$, $\Omega(\epsilon^{-4})$ lower bounds without and with an additional smoothness assumption on $y^*$-aware oracles, respectively. Our results imply that any approach that simulates an algorithm with an $y^*$-aware oracle must suffer the same lower bounds.

math.OC↗

arXiv2024

Stochastic optimization with arbitrary recurrent data sampling

For obtaining optimal first-order convergence guarantee for stochastic optimization, it is necessary to use a recurrent data sampling algorithm that samples every data point with sufficient frequency. Most commonly used data sampling algorithms (e.g., i.i.d., MCMC, random reshuffling) are indeed recurrent under mild assumptions. In this work, we show that for a particular class of stochastic optimization algorithms, we do not need any other property (e.g., independence, exponential mixing, and reshuffling) than recurrence in data sampling algorithms to guarantee the optimal rate of first-order convergence. Namely, using regularized versions of Minimization by Incremental Surrogate Optimization (MISO), we show that for non-convex and possibly non-smooth objective functions, the expected optimality gap converges at an optimal rate $O(n^{-1/2})$ under general recurrent sampling schemes. Furthermore, the implied constant depends explicitly on the `speed of recurrence', measured by the expected amount of time to visit a given data point either averaged (`target time') or supremized (`hitting time') over the current location. We demonstrate theoretically and empirically that convergence can be accelerated by selecting sampling algorithms that cover the data set most effectively. We discuss applications of our general framework to decentralized optimization and distributed non-negative matrix factorization.

math.OC↗

arXiv2024

Four-parameter coalescing ballistic annihilation

In coalescing ballistic annihilation, infinitely many particles move with fixed velocities across the real line and, upon colliding, either mutually annihilate or generate a new particle. We compute the critical density in symmetric three-velocity systems with four-parameter reaction equations.

math.PR↗

arXiv2023

Convergence and complexity of block majorization-minimization for constrained block-Riemannian optimization

Block majorization-minimization (BMM) is a simple iterative algorithm for nonconvex optimization that sequentially minimizes a majorizing surrogate of the objective function in each block coordinate while the other block coordinates are held fixed. We consider a family of BMM algorithms for minimizing smooth nonconvex objectives, where each parameter block is constrained within a subset of a Riemannian manifold. We establish that this algorithm converges asymptotically to the set of stationary points, and attains an $\epsilon$-stationary point within $\widetilde{O}(\epsilon^{-2})$ iterations. In particular, the assumptions for our complexity results are completely Euclidean when the underlying manifold is a product of Euclidean or Stiefel manifolds, although our analysis makes explicit use of the Riemannian geometry. Our general analysis applies to a wide range of algorithms with Riemannian constraints: Riemannian MM, block projected gradient descent, optimistic likelihood estimation, geodesically constrained subspace tracking, robust PCA, and Riemannian CP-dictionary-learning. We experimentally validate that our algorithm converges faster than standard Euclidean algorithms applied to the Riemannian setting.

math.OC↗

arXiv2023

Interpretable Online Network Dictionary Learning for Inferring Long-Range Chromatin Interactions

Dictionary learning (DL) is commonly used in computational biology to tackle ubiquitous clustering problems due to its conceptual simplicity and relatively low computational complexity. However, DL algorithms produce results that lack interpretability and are not optimized for large-scale graph-structured data. We propose a novel DL algorithm called online convex network dictionary learning (online cvxNDL) that can handle extremely large datasets and enables the interpretation of dictionary elements, which serve as cluster representatives, through convex combinations of real measurements. Moreover, the algorithm can be applied to network-structured data via specialized subnetwork sampling techniques. To demonstrate the utility of our approach, we apply cvxNDL on 3D-genome RNAPII ChIA-Drop data to identify important long-range interaction patterns. ChIA-Drop probes higher-order interactions, and produces hypergraphs whose nodes represent genomic fragments. The hyperedges represent observed physical contacts. Our hypergraph model analysis creates an interpretable dictionary of long-range interaction patterns that accurately represent global chromatin physical contact maps. Using dictionary information, one can also associate the contact maps with RNA transcripts and infer cellular functions. Our results offer two key insights. First, we demonstrate that online cvxNDL retains the accuracy of classical DL methods while simultaneously ensuring unique interpretability and scalability. Second, we identify distinct collections of proximal and distal interaction patterns involving chromatin elements shared by related processes across different chromosomes, as well as patterns unique to specific chromosomes. To associate the dictionary elements with biological properties of the corresponding chromatin regions, we employ Gene Ontology enrichment analysis and perform RNA coexpression studies.

q-bio.GN↗

arXiv2023

A latent linear model for nonlinear coupled oscillators on graphs

A system of coupled oscillators on an arbitrary graph is locally driven by the tendency to mutual synchronization between nearby oscillators, but can and often exhibit nonlinear behavior on the whole graph. Understanding such nonlinear behavior has been a key challenge in predicting whether all oscillators in such a system will eventually synchronize. In this paper, we demonstrate that, surprisingly, such nonlinear behavior of coupled oscillators can be effectively linearized in certain latent dynamic spaces. The key insight is that there is a small number of `latent dynamics filters', each with a specific association with synchronizing and non-synchronizing dynamics on subgraphs so that any observed dynamics on subgraphs can be approximated by a suitable linear combination of such elementary dynamic patterns. Taking an ensemble of subgraph-level predictions provides an interpretable predictor for whether the system on the whole graph reaches global synchronization. We propose algorithms based on supervised matrix factorization to learn such latent dynamics filters. We demonstrate that our method performs competitively in synchronization prediction tasks against baselines and black-box classification algorithms, despite its simple and interpretable architecture.

math.DS↗

arXiv2023

Exponentially Convergent Algorithms for Supervised Matrix Factorization

Supervised matrix factorization (SMF) is a classical machine learning method that simultaneously seeks feature extraction and classification tasks, which are not necessarily a priori aligned objectives. Our goal is to use SMF to learn low-rank latent factors that offer interpretable, data-reconstructive, and class-discriminative features, addressing challenges posed by high-dimensional data. Training SMF model involves solving a nonconvex and possibly constrained optimization with at least three blocks of parameters. Known algorithms are either heuristic or provide weak convergence guarantees for special cases. In this paper, we provide a novel framework that 'lifts' SMF as a low-rank matrix estimation problem in a combined factor space and propose an efficient algorithm that provably converges exponentially fast to a global minimizer of the objective with arbitrary initialization under mild assumptions. Our framework applies to a wide range of SMF-type problems for multi-class classification with auxiliary features. To showcase an application, we demonstrate that our algorithm successfully identified well-known cancer-associated gene groups for various cancers.

stat.ML↗