arXiv subjects

Lexing Ying

Publications and source records attributed to Lexing Ying.

At least 37 records · Page 2Linked to original sources

arXiv2025

Variational inference and density estimation with non-negative tensor train

This work proposes an efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format. The two settings we consider are variational inference and density estimation, whereby one has access to either the unnormalized analytic formula of the distribution or the samples generated from the distribution. In particular, the compression is done through a two-stage approach. In the first stage, we use existing subroutines to encode the distribution function in a tensor train format. In the second stage, we use an NTT ansatz to fit the obtained tensor train. For the NTT fitting procedure, we use a log barrier term to ensure the positivity of each tensor component, and then utilize a second-order alternating minimization scheme to accelerate convergence. In practice, we observe that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed. Through challenging numerical experiments, we show that our approach can accurately compress target distribution functions.

math.NA↗

arXiv2025

Blind free deconvolution over one-parameter sparse families via eigenmatrix

This note considers the blind free deconvolution problems of sparse spectral measures from one-parameter families. These problems pose significant challenges since they involve nonlinear sparse recovery. The main technical tool is the eigenmatrix method for solving unstructured sparse recovery problems. The key idea is to turn the nonlinear inverse problem into a linear inverse problem by leveraging the R-transform for free addition and the S-transform for free product. The resulting linear problem is solved with the eigenmatrix method tailored to the domain of the parametric family. Numerical results are provided for both the additive and multiplicative free deconvolutions.

math.NA↗

arXiv2025

Solving high-dimensional Hamilton-Jacobi-Bellman equation with functional hierarchical tensor

This work proposes a novel numerical scheme for solving the high-dimensional Hamilton-Jacobi-Bellman equation with a functional hierarchical tensor ansatz. We consider the setting of stochastic control, whereby one applies control to a particle under Brownian motion. In particular, the existence of diffusion presents a new challenge to conventional tensor network methods for deterministic optimal control. To overcome the difficulty, we use a general regression-based formulation where the loss term is the Bellman consistency error combined with a Sobolev-type penalization term. We propose two novel sketching-based subroutines for obtaining the tensor-network approximation to the action-value functions and the value functions, which greatly accelerate the convergence for the subsequent regression phase. We apply the proposed approach successfully to two challenging control problems with Ginzburg-Landau potential in 1D and 2D with 64 variables.

math.NA↗

arXiv2025

COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework

In LLM alignment and many other ML applications, one often faces the Multi-Objective Fine-Tuning (MOFT) problem, i.e., fine-tuning an existing model with datasets labeled w.r.t. different objectives simultaneously. To address the challenge, we propose a Conditioned One-Shot fine-tuning framework (COS-DPO) that extends the Direct Preference Optimization technique, originally developed for efficient LLM alignment with preference data, to accommodate the MOFT settings. By direct conditioning on the weight across auxiliary objectives, our Weight-COS-DPO method enjoys an efficient one-shot training process for profiling the Pareto front and is capable of achieving comprehensive trade-off solutions even in the post-training stage. Based on our theoretical findings on the linear transformation properties of the loss function, we further propose the Temperature-COS-DPO method that augments the temperature parameter to the model input, enhancing the flexibility of post-training control over the trade-offs between the main and auxiliary objectives. We demonstrate the effectiveness and efficiency of the COS-DPO framework through its applications to various tasks, including the Learning-to-Rank (LTR) and LLM alignment tasks, highlighting its viability for large-scale ML deployments.

cs.LG↗

arXiv2025

Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach

Diffusion models (DMs) have proven to be effective in modeling high-dimensional distributions, leading to their widespread adoption for representing complex priors in Bayesian inverse problems (BIPs). However, current DM-based posterior sampling methods proposed for solving common BIPs rely on heuristic approximations to the generative process. To exploit the generative capability of DMs and avoid the usage of such approximations, we propose an ensemble-based algorithm that performs posterior sampling without the use of heuristic approximations. Our algorithm is motivated by existing works that combine DM-based methods with the sequential Monte Carlo (SMC) method. By examining how the prior evolves through the diffusion process encoded by the pre-trained score function, we derive a modified partial differential equation (PDE) governing the evolution of the corresponding posterior distribution. This PDE includes a modified diffusion term and a reweighting term, which can be simulated via stochastic weighted particle methods. Theoretically, we prove that the error between the true posterior distribution can be bounded in terms of the training error of the pre-trained score function and the number of particles in the ensemble. Empirically, we validate our algorithm on several inverse problems in imaging to show that our method gives more accurate reconstructions compared to existing DM-based methods.

cs.LG↗

arXiv2025

On a spherically lifted spin model at finite temperature

We investigate an $n$-vector model over $k$ sites with generic pairwise interactions and spherical constraints. The model is a lifting of the Ising model whereby the support of the spin is lifted to a hypersphere. We show that the $n$-vector model converges to a limiting distribution at a rate of $n^{-1/2 + o(1)}$. We show that the limiting distribution for $n \to \infty$ is determined by the solution of an equality-constrained maximization task over positive definite matrices. We prove that the obtained maximal value and maximizer, respectively, give rise to the free energy and correlation function of the limiting distribution. In the finite temperature regime, the maximization task is a log-determinant regularization of the semidefinite program (SDP) in the Goemans-Williamson algorithm. Moreover, the inverse temperature determines the regularization strength, with the zero temperature limit converging to the SDP in Goemans-Williamson. Our derivation draws a curious connection between the semidefinite relaxation of integer programming and the spherical lifting of sampling on a hypercube. To the authors' best knowledge, this work is the first to solve the setting of fixed $k$ and infinite $n$ under unstructured pairwise interactions.

math.PR↗

arXiv2025

Inverse nonlinear fast Fourier transform on SU(2) with applications to quantum signal processing

The nonlinear Fourier transform (NLFT) extends the classical Fourier transform by replacing addition with matrix multiplication. While the NLFT on $\mathrm{SU}(1,1)$ has been widely studied, its $\mathrm{SU}(2)$ variant has only recently attracted attention due to emerging applications in quantum signal processing (QSP) and quantum singular value transformation (QSVT). In this paper, we investigate the inverse NLFT on $\mathrm{SU}(2)$ and establish the numerical stability of the layer stripping algorithm for the first time under suitable conditions. Furthermore, we develop a fast and numerically stable algorithm, called inverse nonlinear fast Fourier transform, for performing inverse NLFT with near-linear complexity. This algorithm is applicable to computing phase factors for both QSP and the generalized QSP (GQSP).

quant-ph↗

arXiv2025

Initialization and training of matrix product state probabilistic models

Modeling probability distributions via the wave function of a quantum state is central to quantum-inspired generative modeling and quantum state tomography (QST). We investigate a common failure mode in training randomly initialized matrix product states (MPS) using gradient descent. The results show that the trained MPS models do not accurately predict the strong interactions between boundary sites in periodic spin chain models. In the case of the Born machine algorithm, we further identify a causality trap, where the trained MPS models resemble causal models that ignore the non-local correlations in the true distribution. We propose two complementary strategies to overcome the training failure -- one through optimization and one through initialization. First, we develop a natural gradient descent (NGD) method, which approximately simulates the gradient flow on tensor manifolds and significantly enhances training efficiency. Numerical experiments show that NGD avoids local minima in both Born machines and in general MPS tomography. Remarkably, we show that NGD with line search can converge to the global minimum in only a few iterations. Second, for the BM algorithm, we introduce a warm-start initialization based on the TTNS-Sketch algorithm. We show that gradient descent under a warm initialization does not encounter the causality trap and admits rapid convergence to the ground truth.

math.NA↗

arXiv2025

How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework

Discrete diffusion models have gained increasing attention for their ability to model complex distributions with tractable sampling and inference. However, the error analysis for discrete diffusion models remains less well-understood. In this work, we propose a comprehensive framework for the error analysis of discrete diffusion models based on Lévy-type stochastic integrals. By generalizing the Poisson random measure to that with a time-independent and state-dependent intensity, we rigorously establish a stochastic integral formulation of discrete diffusion models and provide the corresponding change of measure theorems that are intriguingly analogous to Itô integrals and Girsanov's theorem for their continuous counterparts. Our framework unifies and strengthens the current theoretical results on discrete diffusion models and obtains the first error bound for the $τ$-leaping scheme in KL divergence. With error sources clearly identified, our analysis gives new insight into the mathematical properties of discrete diffusion models and offers guidance for the design of efficient and accurate algorithms for real-world discrete diffusion model applications.

cs.LG↗

arXiv2025

Wavelet-based density sketching with functional hierarchical tensor

We introduce the functional hierarchical tensor under a wavelet basis (FHT-W) ansatz for high-dimensional density estimation in lattice models. Recently, the functional tensor network has emerged as a suitable candidate for density estimation due to its ability to calculate the normalization constant exactly, a defining feature not enjoyed by neural network alternatives such as energy-based models or diffusion models. While current functional tensor network models show good performance for lattice models with weak or moderate couplings, we show that they face significant model capacity constraints when applied to lattice models with strong coupling. To address this issue, this work proposes to perform density estimation on the lattice model under a wavelet transformation. Motivated by the literature on scale separation, we perform iterative wavelet coarsening to separate the lattice model into different scales. Based on this multiscale structure, we design a new functional hierarchical tensor ansatz using a hierarchical tree topology, whereby information on the finer scale is further away from the root node of the tree. Our experiments show that the numerical rank of typical lattice models is significantly lower under appropriate wavelet transformation. Furthermore, we show that our proposed model allows one to model challenging Gaussian field models and Ginzburg-Landau models.

math.NA↗

arXiv2025

Approximation of High-Dimensional Gibbs Distributions with Functional Hierarchical Tensors

The numerical representation of high-dimensional Gibbs distributions is challenging due to the curse of dimensionality manifesting through the intractable normalization constant calculations. This work addresses this challenge by performing a particle-based high-dimensional parametric density estimation subroutine, and the input to the subroutine is Gibbs samples generated by leveraging advanced sampling techniques. Specifically, to generate Gibbs samples, we employ ensemble-based annealed importance sampling, a population-based approach for sampling multimodal distributions. These samples are then processed using functional hierarchical tensor sketching, a tensor-network-based density estimation method for high-dimensional distributions, to obtain the numerical representation of the Gibbs distribution. We successfully apply the proposed approach to complex Ginzburg-Landau models with hundreds of variables. In particular, we show that the approach proposed is successful at addressing the metastability issue under difficult numerical cases.

math.NA↗

arXiv2025

Sparse free deconvolution under unknown noise level via eigenmatrix

This note considers the spectral estimation problems of sparse spectral measures under unknown noise levels. The main technical tool is the eigenmatrix method for solving unstructured sparse recovery problems. When the noise level is determined, the free deconvolution reduces the problem to an unstructured sparse recovery problem to which the eigenmatrix method can be applied. To determine the unknown noise level, we propose an optimization problem based on the singular values of an intermediate matrix of the eigenmatrix method. Numerical results are provided for both the additive and multiplicative free deconvolutions.

math.NA↗

arXiv2024

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow

Multi-objective optimization (MOO) aims to optimize multiple, possibly conflicting objectives with widespread applications. We introduce a novel interacting particle method for MOO inspired by molecular dynamics simulations. Our approach combines overdamped Langevin and birth-death dynamics, incorporating a "dominance potential" to steer particles toward global Pareto optimality. In contrast to previous methods, our method is able to relocate dominated particles, making it particularly adept at managing Pareto fronts of complicated geometries. Our method is also theoretically grounded as a Wasserstein-Fisher-Rao gradient flow with convergence guarantees. Extensive experiments confirm that our approach outperforms state-of-the-art methods on challenging synthetic and real-world datasets.

cs.LG↗

arXiv2024

Ensemble-Based Annealed Importance Sampling

Sampling from a multimodal distribution is a fundamental and challenging problem in computational science and statistics. Among various approaches proposed for this task, one popular method is Annealed Importance Sampling (AIS). In this paper, we propose an ensemble-based version of AIS by combining it with population-based Monte Carlo methods to improve its efficiency. By keeping track of an ensemble instead of a single particle along some continuation path between the starting distribution and the target distribution, we take advantage of the interaction within the ensemble to encourage the exploration of undiscovered modes. Specifically, our main idea is to utilize either the snooker algorithm or the genetic algorithm used in Evolutionary Monte Carlo. We discuss how the proposed algorithm can be implemented and derive a partial differential equation governing the evolution of the ensemble under the continuous time and mean-field limit. We also test the efficiency of the proposed algorithm on various continuous and discrete distributions.

stat.CO↗

arXiv2024

Laplace transform based quantum eigenvalue transformation via linear combination of Hamiltonian simulation

Eigenvalue transformations, which include solving time-dependent differential equations as a special case, have a wide range of applications in scientific and engineering computation. While quantum algorithms for singular value transformations are well studied, eigenvalue transformations are distinct, especially for non-normal matrices. We propose an efficient quantum algorithm for performing a class of eigenvalue transformations that can be expressed as a certain type of matrix Laplace transformation. This allows us to significantly extend the recently developed linear combination of Hamiltonian simulation (LCHS) method [An, Liu, Lin, Phys. Rev. Lett. 131, 150603, 2023; An, Childs, Lin, arXiv:2312.03916] to represent a wider class of eigenvalue transformations, such as powers of the matrix inverse, $A^{-k}$, and the exponential of the matrix inverse, $e^{-A^{-1}}$. The latter can be interpreted as the solution of a mass-matrix differential equation of the form $A u'(t)=-u(t)$. We demonstrate that our eigenvalue transformation approach can solve this problem without explicitly inverting $A$, reducing the computational complexity.

quant-ph↗

arXiv2024

Fast Phase Factor Finding for Quantum Signal Processing

This paper presents two efficient and stable algorithms for recovering phase factors in quantum signal processing (QSP), a crucial component of many quantum algorithms. The first algorithm, the ``Half Cholesky" method, which is based on nonlinear Fourier analysis and fast solvers for structured matrices, demonstrates robust performance across all regimes. The second algorithm, ``Fast Fixed Point Iteration," provides even greater efficiency in the non-fully-coherent regime. Both theoretical analysis and numerical experiments demonstrate the significant advantages of these new methods over all existing approaches.

quant-ph↗

arXiv2024

Quantum Multiple Eigenvalue Gaussian filtered Search: an efficient and versatile quantum phase estimation method

Quantum phase estimation is one of the most powerful quantum primitives. This work proposes a new approach for the problem of multiple eigenvalue estimation: Quantum Multiple Eigenvalue Gaussian filtered Search (QMEGS). QMEGS leverages the Hadamard test circuit structure and only requires simple classical postprocessing. QMEGS is the first algorithm to simultaneously satisfy the following two properties: (1) It can achieve the Heisenberg-limited scaling without relying on any spectral gap assumption. (2) With a positive energy gap and additional assumptions on the initial state, QMEGS can estimate all dominant eigenvalues to $ε$ accuracy utilizing a significantly reduced circuit depth compared to the standard quantum phase estimation algorithm. In the most favorable scenario, the maximal runtime can be reduced to as low as $\log(1/ε)$. This implies that QMEGS serves as an efficient and versatile approach, achieving the best-known results for both gapped and gapless systems. Numerical results validate the efficiency of our proposed algorithm in various regimes.

quant-ph↗

arXiv2024

Tangent differential privacy

Differential privacy is a framework for protecting the identity of individual data points in the decision-making process. In this note, we propose a new form of differential privacy called tangent differential privacy. Compared with the usual differential privacy that is defined uniformly across data distributions, tangent differential privacy is tailored towards a specific data distribution of interest. It also allows for general distribution distances such as total variation distance and Wasserstein distance. In the case of risk minimization, we show that entropic regularization guarantees tangent differential privacy under rather general conditions on the risk function.

cs.LG↗