arXiv subjects

Kaylee Yingxi Yang

Publications and source records attributed to Kaylee Yingxi Yang.

2 recordsLinked to original sources

arXiv2026

Convergence of the Inexact Langevin Algorithm in KL Divergence with Application to Score-based Generative Models

Motivated by the increasingly popular Score-based Generative Modeling (SGM), we study the Inexact Langevin Dynamics (ILD) and Inexact Langevin Algorithm (ILA) where a score function estimate is used in place of the exact score. We establish {\em stable} biased convergence guarantees in terms of the Kullback-Leibler (KL) divergence. To achieve these guarantees, we impose two key assumptions: 1) the target distribution satisfies the log-Sobolev inequality, and 2) the error of score estimator exhibits a sub-Gaussian tail, referred to as Moment Generating Function (MGF) error assumption. Under the stronger $L^\infty$ score error assumption, we obtain a stable convergence bound in Rényi divergence. We also generalize the proof technique to SGM, and derive a stable convergence bound in KL divergence. In addition, we explore the question of how to obtain a provably accurate score estimator. We demonstrate that a simple estimator based on kernel density estimation fulfills the MGF error assumption for sub-Gaussian target distributions, at the population level.

cs.LG↗

arXiv2024

Optimal score estimation via empirical Bayes smoothing

We study the problem of estimating the score function of an unknown probability distribution $ρ^*$ from $n$ independent and identically distributed observations in $d$ dimensions. Assuming that $ρ^*$ is subgaussian and has a Lipschitz-continuous score function $s^*$, we establish the optimal rate of $\tilde Θ(n^{-\frac{2}{d+4}})$ for this estimation problem under the loss function $\|\hat s - s^*\|^2_{L^2(ρ^*)}$ that is commonly used in the score matching literature, highlighting the curse of dimensionality where sample complexity for accurate score estimation grows exponentially with the dimension $d$. Leveraging key insights in empirical Bayes theory as well as a new convergence rate of smoothed empirical distribution in Hellinger distance, we show that a regularized score estimator based on a Gaussian kernel attains this rate, shown optimal by a matching minimax lower bound. We also discuss extensions to estimating $β$-Hölder continuous scores with $β\leq 1$, as well as the implication of our theory on the sample complexity of score-based generative models.

math.ST↗