arXiv subjects

Su-Yun Huang

Publications and source records attributed to Su-Yun Huang.

At least 19 recordsLinked to original sources

arXiv2026

Function-On-Function Regression Through Separable Neural Operators

This paper investigates the estimation of the regression operator in function-on-function regression models. While traditional research has predominantly focused on linear models or their immediate nonlinear extensions, we propose a neural operator approach to accommodate general regression operators under mild smoothness assumptions. Operator learning has emerged as an active area of machine learning, particularly for solving physical models governed by partial differential equations. Using this paradigm, our methodology introduces the separable neural operator, a neural-operator architecture that represents the regression operator through input-dependent coefficient functions and output-dependent basis functions. Beyond adapting this architecture to the regression operator estimation problem, we establish the consistency of the estimator under relatively mild smoothness and sampling conditions, allowing functional data to be observed on dense, possibly irregular, discrete grids. We also apply the proposed approach to the BGC Argo data and demonstrate its potential for oceanographic research.

math.ST↗

arXiv2026

Blurring Mean Shift for Clustering Functional Data: A Scalable Algorithm and Convergence Analysis

This paper extends the blurring mean shift algorithm from vector-valued data to functional data, enabling effective clustering in infinite-dimensional settings without requiring specification of the number of clusters. To address the computational challenges posed by large-scale datasets, we introduce a fast stochastic variant that significantly reduces computational complexity. We provide a rigorous convergence analysis for the full blurring functional mean shift procedure, establishing theoretical guarantees for its iterative behavior. For the stochastic variant, we provide partial theoretical justification by showing that, when the subset size is sufficiently large, its one-step update is well approximated by the corresponding update of the full algorithm. The proposed method is demonstrated through real-data applications, including hourly Taiwan PM$_{2.5}$ measurements and Argo oceanographic profiles. Our key contributions include: (1) extending the blurring mean shift algorithm to functional data in a Hilbert-space setting; (2) developing a scalable stochastic variant based on random partitioning for large-scale data; (3) establishing convergence results for the full blurring functional mean shift algorithm; and (4) demonstrating the scalability and practical usefulness of the proposed method through simulation and real-data applications.

stat.ME↗

arXiv2026

Generative and Nonparametric Approaches for Conditional Distribution Estimation: Methods, Perspectives, and Comparative Evaluations

The inference of conditional distributions is a fundamental problem in statistics, essential for prediction, uncertainty quantification, and probabilistic modeling. A wide range of methodologies have been developed for this task. This article reviews and compares several representative approaches spanning classical nonparametric methods and modern generative models. We begin with the single-index method of Hall and Yao (2005), which estimates the conditional distribution through a dimension-reducing index and nonparametric smoothing of the resulting one-dimensional cumulative conditional distribution function. We then examine the basis-expansion approaches, including FlexCode (Izbicki and Lee, 2017) and DeepCDE (Dalmasso et al., 2020), which convert conditional density estimation into a set of nonparametric regression problems. In addition, we discuss two recent generative simulation-based methods that leverage modern deep generative architectures: the generative conditional distribution sampler (Zhou et al., 2023) and the conditional denoising diffusion probabilistic model (Fu et al., 2024; Yang et al., 2025). A systematic numerical comparison of these approaches is provided using a unified evaluation framework that ensures fairness and reproducibility. The performance metrics used for the estimated conditional distribution include the mean-squared errors of conditional mean and standard deviation, as well as the Wasserstein distance. We also discuss their flexibility and computational costs, highlighting the distinct advantages and limitations of each approach.

stat.ML↗

arXiv2025

The $ϕ$-PCA Framework: A Unified and Efficiency-Preserving Approach with Robust Variants

Principal component analysis (PCA) is a fundamental tool in multivariate statistics, yet its sensitivity to outliers and limitations in distributed environments restrict its effectiveness in modern large-scale applications. To address these challenges, we introduce the $ϕ$-PCA framework which provides a unified formulation of robust and distributed PCA. The class of $ϕ$-PCA methods retains the asymptotic efficiency of standard PCA, while aggregating multiple local estimates using a proper $ϕ$ function enhances ordering-robustness, leading to more accurate eigensubspace estimation under contamination. Notably, the harmonic mean PCA (HM-PCA), corresponding to the choice $ϕ(u)=u^{-1}$, achieves optimal ordering-robustness and is recommended for practical use. Theoretical results further show that robustness increases with the number of partitions, a phenomenon seldom explored in the literature on robust or distributed PCA. Altogether, the partition-aggregation principle underlying $ϕ$-PCA offers a general strategy for developing robust and efficiency-preserving methodologies applicable to both robust and distributed data analysis.

stat.ME↗

arXiv2025

On the efficiency-loss free ordering-robustness of product-PCA

This article studies the robustness of the eigenvalue ordering, an important issue when estimating the leading eigen-subspace by principal component analysis (PCA). In Yata and Aoshima (2010), cross-data-matrix PCA (CDM-PCA) was proposed and shown to have smaller bias than PCA in estimating eigenvalues. While CDM-PCA has the potential to achieve better estimation of the leading eigen-subspace than the usual PCA, its robustness is not well recognized. In this article, we first develop a more stable variant of CDM-PCA, which we call product-PCA (PPCA), that provides a more convenient formulation for theoretical investigation. Secondly, we prove that, in the presence of outliers, PPCA is more robust than PCA in maintaining the correct ordering of leading eigenvalues. The robustness gain in PPCA comes from the random data partition, and it does not rely on a data down-weighting scheme as most robust statistical methods do. This enables us to establish the surprising finding that, when there are no outliers, PPCA and PCA share the same asymptotic distribution. That is, the robustness gain of PPCA in estimating the leading eigen-subspace has no efficiency loss in comparison with PCA. Simulation studies and a face data example are presented to show the merits of PPCA. In conclusion, PPCA has a good potential to replace the role of the usual PCA in real applications whether outliers are present or not.

stat.ME↗

arXiv2024

On the asymptotic properties of product-PCA under the high-dimensional setting

Principal component analysis (PCA) is a widely used dimension reduction method, but its performance is known to be non-robust to outliers. Recently, product-PCA (PPCA) has been shown to possess the efficiency-loss free ordering-robustness property: (i) in the absence of outliers, PPCA and PCA share the same asymptotic distributions; (ii), in the presence of outliers, PPCA is more ordering-robust than PCA in estimating the leading eigenspace. PPCA is thus different from the conventional robust PCA methods, and may deserve further investigations. In this article, we study the high-dimensional statistical properties of the PPCA eigenvalues via the techniques of random matrix theory. In particular, we derive the critical value for being distant spiked eigenvalues, the limiting values of the sample spiked eigenvalues, and the limiting spectral distribution of PPCA. Similar to the case of PCA, the explicit forms of the asymptotic properties of PPCA become available under the special case of the simple spiked model. These results enable us to more clearly understand the superiorities of PPCA in comparison with PCA. Numerical studies are conducted to verify our results.

math.ST↗

arXiv2024

TensorProjection Layer: A Tensor-Based Dimension Reduction Method in Deep Neural Networks

In this paper, we propose a dimension reduction method specifically designed for tensor-structured feature data in deep neural networks. The method is implemented as a hidden layer, called the TensorProjection layer, which transforms input tensors into output tensors with reduced dimensions through mode-wise projections. The projection directions are treated as model parameters of the layer and are optimized during model training. Our method can serve as an alternative to pooling layers for summarizing image data, or to convolutional layers as a technique for reducing the number of channels. We conduct experiments on tasks such as medical image classification and segmentation, integrating the TensorProjection layer into commonly used baseline architectures to evaluate its effectiveness. Numerical experiments indicate that the proposed method can outperform traditional downsampling methods, such as pooling layers, in our tasks, suggesting it as a promising alternative for feature summarization.

stat.ML↗

arXiv2024

A Generalized Mean Approach for Distributed-PCA

Principal component analysis (PCA) is a widely used technique for dimension reduction. As datasets continue to grow in size, distributed-PCA (DPCA) has become an active research area. A key challenge in DPCA lies in efficiently aggregating results across multiple machines or computing nodes due to computational overhead. Fan et al. (2019) introduced a pioneering DPCA method to estimate the leading rank-$r$ eigenspace, aggregating local rank-$r$ projection matrices by averaging. However, their method does not utilize eigenvalue information. In this article, we propose a novel DPCA method that incorporates eigenvalue information to aggregate local results via the matrix $β$-mean, which we call $β$-DPCA. The matrix $β$-mean offers a flexible and robust aggregation method through the adjustable choice of $β$ values. Notably, for $β=1$, it corresponds to the arithmetic mean; for $β=-1$, the harmonic mean; and as $β\to 0$, the geometric mean. Moreover, the matrix $β$-mean is shown to associate with the matrix $β$-divergence, a subclass of the Bregman matrix divergence, to support the robustness of $β$-DPCA. We also study the stability of eigenvector ordering under eigenvalue perturbation for $β$-DPCA. The performance of our proposal is evaluated through numerical studies.

stat.ML↗

arXiv2024

Information Criterion-Based Rank Estimation Methods for Factor Analysis: A Unified Selection Consistency Theorem and Numerical Comparison

Over the years, numerous rank estimators for factor models have been proposed in the literature. This article focuses on information criterion-based rank estimators and investigates their consistency in rank selection. The gap conditions serve as necessary and sufficient conditions for rank estimators to achieve selection consistency under the general assumptions of random matrix theory. We establish a unified theorem on selection consistency, presenting the gap conditions for information criterion-based rank estimators with a unified formulation. To validate the theorem's assertion that rank selection consistency is solely determined by the gap conditions, we conduct extensive numerical simulations across various settings. Additionally, we undertake supplementary simulations to explore the strengths and limitations of information criterion-based estimators by comparing them with other types of rank estimators.

math.ST↗

arXiv2022

Robust self-tuning semiparametric PCA for contaminated elliptical distribution

Principal component analysis (PCA) is one of the most popular dimension reduction methods. The usual PCA is known to be sensitive to the presence of outliers, and thus many robust PCA methods have been developed. Among them, the Tyler's M-estimator is shown to be the most robust scatter estimator under the elliptical distribution. However, when the underlying distribution is contaminated and deviates from ellipticity, Tyler's M-estimator might not work well. In this article, we apply the semiparametric theory to propose a robust semiparametric PCA. The merits of our proposal are twofold. First, it is robust to heavy-tailed elliptical distributions as well as robust to non-elliptical outliers. Second, it pairs well with a data-driven tuning procedure, which is based on active ratio and can adapt to different degrees of data outlyingness. Theoretical properties are derived, including the influence functions for various statistical functionals and asymptotic normality. Simulation studies and a data analysis demonstrate the superiority of our method.

stat.ME↗

arXiv2021

Two-stage dimension reduction for noisy high-dimensional images and application to Cryogenic Electron Microscopy

Principal component analysis (PCA) is arguably the most widely used dimension-reduction method for vector-type data. When applied to a sample of images, PCA requires vectorization of the image data, which in turn entails solving an eigenvalue problem for the sample covariance matrix. We propose herein a two-stage dimension reduction (2SDR) method for image reconstruction from high-dimensional noisy image data. The first stage treats the image as a matrix, which is a tensor of order 2, and uses multilinear principal component analysis (MPCA) for matrix rank reduction and image denoising. The second stage vectorizes the reduced-rank matrix and achieves further dimension and noise reduction. Simulation studies demonstrate excellent performance of 2SDR, for which we also develop an asymptotic theory that establishes consistency of its rank selection. Applications to cryo-EM (cryogenic electronic microscopy), which has revolutionized structural biology, organic and medical chemistry, cellular and molecular physiology in the past decade, are also provided and illustrated with benchmark cryo-EM datasets. Connections to other contemporaneous developments in image reconstruction and high-dimensional statistical inference are also discussed.

eess.IV↗

arXiv2020

A generalized information criterion for high-dimensional PCA rank selection

Principal component analysis (PCA) is the most commonly used statistical procedure for dimension reduction. An important issue for applying PCA is to determine the rank, which is the number of dominant eigenvalues of the covariance matrix. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) are among the most widely used rank selection methods. Both use the number of free parameters for assessing model complexity. In this work, we adopt the generalized information criterion (GIC) to propose a new method for PCA rank selection under the high-dimensional framework. The GIC model complexity takes into account the sizes of covariance eigenvalues and can be better adaptive to practical applications. Asymptotic properties of GIC are derived and the selection consistency is established under the generalized spiked covariance model.

stat.ME↗

arXiv2020

A Consistency Theorem for Randomized Singular Value Decomposition

The singular value decomposition (SVD) and the principal component analysis are fundamental tools and probably the most popular methods for data dimension reduction. The rapid growth in the size of data matrices has lead to a need for developing efficient large-scale SVD algorithms. Randomized SVD was proposed, and its potential was demonstrated for computing a low-rank SVD (Rokhlin et al., 2009). In this article, we provide a consistency theorem for the randomized SVD algorithm and a numerical example to show how the random projections to low dimension affect the consistency.

math.ST↗

arXiv2019

Random Partitioning and Distribution-based Thresholding for Iterative Variable Screening in High Dimensions

In big data analysis, a simple task such as linear regression can become very challenging as the variable dimension $p$ grows. As a result, variable screening is inevitable in many scientific studies. In recent years, randomized algorithms have become a new trend and are playing an increasingly important role for large scale data analysis. In this article, we combine the ideas of variable screening and random partitioning to propose a new iterative variable screening method. For moderate sized $p$ of order $O(n^{2-δ})$, we propose a basic algorithm that adopts a distribution-based thresholding rule. For very large $p$, we further propose a two-stage procedure. This two-stage procedure first performs a random partitioning to divide predictors into subsets of manageable size of order $O(n^{2-δ})$ for variable screening, where $δ>0$ can be an arbitrarily small positive number. Random partitioning is repeated a few times. Next, the final estimate of variable subset is obtained by integrating results obtained from multiple random partitions. Simulation studies show that our method works well and outperforms some renowned competitors. Real data applications are presented. Our algorithms are able to handle predictors in the size of millions.

stat.ME↗

arXiv2017

Sufficient Dimension Reduction via Random-Partitions for Large-p-Small-n Problem

Sufficient dimension reduction (SDR) is continuing an active research field nowadays for high dimensional data. It aims to estimate the central subspace (CS) without making distributional assumption. To overcome the large-$p$-small-$n$ problem we propose a new approach for SDR. Our method combines the following ideas for high dimensional data analysis: (1) Randomly partition the covariates into subsets and use distance correlation (DC) to construct a sketch of envelope subspace with low dimension. (2) Obtain a sketch of the CS by applying conventional SDR method within the constructed envelope subspace. (3) Repeat the above two steps for a few times and integrate these multiple sketches to form the final estimate of the CS. We name the proposed SDR procedure "integrated random-partition SDR (iRP-SDR)". Comparing with existing methods, iRP-SDR is less affected by the selection of tuning parameters. Moreover, the estimation procedure of iRP-SDR does not involve the determination of the structural dimension until at the last stage, which makes the method more robust in a high-dimensional setting. Asymptotic properties of iRP-SDR are also established. The advantageous performance of the proposed method is demonstrated via simulation studies and the EEG data analysis.

stat.ME↗

arXiv2017

Robust mislabel logistic regression without modeling mislabel probabilities

Logistic regression is among the most widely used statistical methods for linear discriminant analysis. In many applications, we only observe possibly mislabeled responses. Fitting a conventional logistic regression can then lead to biased estimation. One common resolution is to fit a mislabel logistic regression model, which takes into consideration of mislabeled responses. Another common method is to adopt a robust M-estimation by down-weighting suspected instances. In this work, we propose a new robust mislabel logistic regression based on gamma-divergence. Our proposal possesses two advantageous features: (1) It does not need to model the mislabel probabilities. (2) The minimum gamma-divergence estimation leads to a weighted estimating equation without the need to subtract any bias correction term, i.e., it is automatically bias corrected. These properties make the proposed gamma-logistic regression more robust in model fitting and more intuitive for model interpretation through a simple weighting scheme. Our method is also easy to implement, and two types of algorithms are included. Simulation and real data application results are presented to demonstrate the performance of gamma-logistic.

stat.AP↗

arXiv2016

Integrating multiple random sketches for singular value decomposition

The singular value decomposition (SVD) of large-scale matrices is a key tool in data analytics and scientific computing. The rapid growth in the size of matrices further increases the need for developing efficient large-scale SVD algorithms. Randomized SVD based on one-time sketching has been studied, and its potential has been demonstrated for computing a low-rank SVD. Instead of exploring different single random sketching techniques, we propose a Monte Carlo type integrated SVD algorithm based on multiple random sketches. The proposed integration algorithm takes multiple random sketches and then integrates the results obtained from the multiple sketched subspaces. So that the integrated SVD can achieve higher accuracy and lower stochastic variations. The main component of the integration is an optimization problem with a matrix Stiefel manifold constraint. The optimization problem is solved using Kolmogorov-Nagumo-type averages. Our theoretical analyses show that the singular vectors can be induced by population averaging and ensure the consistencies between the computed and true subspaces and singular vectors. Statistical analysis further proves a strong Law of Large Numbers and gives a rate of convergence by the Central Limit Theorem. Preliminary numerical results suggest that the proposed integrated SVD algorithm is promising.

math.NA↗

arXiv2015

Functional Inverse Regression in an Enlarged Dimension Reduction Space

We consider an enlarged dimension reduction space in functional inverse regression. Our operator and functional analysis based approach facilitates a compact and rigorous formulation of the functional inverse regression problem. It also enables us to expand the possible space where the dimension reduction functions belong. Our formulation provides a unified framework so that the classical notions, such as covariance standardization, Mahalanobis distance, SIR and linear discriminant analysis, can be naturally and smoothly carried out in our enlarged space. This enlarged dimension reduction space also links to the linear discriminant space of Gaussian measures on a separable Hilbert space.

math.ST↗