Searcharxiv⌕ Search

arXiv subjects

Yu-Chun Kao

Publications and source records attributed to Yu-Chun Kao.

3 recordsLinked to original sources
arXiv2025

Optimal convex $M$-estimation via score matching

In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal asymptotic variance in the downstream estimation of the regression coefficients. At the population level, the negative derivative of the optimal convex loss is the best decreasing approximation of the derivative of the log-density of the noise distribution. This motivates a fitting process via a nonparametric extension of score matching, corresponding to a log-concave projection of the noise distribution with respect to the Fisher divergence. At the sample level, our semiparametric estimator is computationally efficient, and we prove that it attains the minimal asymptotic covariance among all convex $M$-estimators. As an example of a non-log-concave setting, the optimal convex loss function for Cauchy errors is Huber-like, and our procedure yields asymptotic efficiency greater than $0.87$ relative to the maximum likelihood estimator of the regression coefficients that uses oracle knowledge of this error distribution. In this sense, we provide robustness and facilitate computation without sacrificing much statistical efficiency. Numerical experiments using our accompanying R package 'asm' confirm the practical merits of our proposal.

math.ST↗
arXiv2023

Choosing the $p$ in $L_p$ loss: rate adaptivity on the symmetric location problem

Given univariate random variables $Y_1, \ldots, Y_n$ with the $\text{Uniform}(θ_0 - 1, θ_0 + 1)$ distribution, the sample midrange $\frac{Y_{(n)}+Y_{(1)}}{2}$ is the MLE for $θ_0$ and estimates $θ_0$ with error of order $1/n$, which is much smaller compared with the $1/\sqrt{n}$ error rate of the usual sample mean estimator. However, the sample midrange performs poorly when the data has say the Gaussian $N(θ_0, 1)$ distribution, with an error rate of $1/\sqrt{\log n}$. In this paper, we propose an estimator of the location $θ_0$ with a rate of convergence that can, in many settings, adapt to the underlying distribution which we assume to be symmetric around $θ_0$ but is otherwise unknown. When the underlying distribution is compactly supported, we show that our estimator attains a rate of convergence of $n^{-\frac{1}α}$ up to polylog factors, where the rate parameter $α$ can take on any value in $(0, 2]$ and depends on the moments of the underlying distribution. Our estimator is formed by the $\ell^γ$-center of the data, for a $γ\geq2$ chosen in a data-driven way -- by minimizing a criterion motivated by the asymptotic variance. Our approach can be directly applied to the regression setting where $θ_0$ is a function of observed features and motivates the use of $\ell^γ$ loss function for $γ> 2$ in certain settings.

math.ST↗