arXiv subjects

Sungkyu Jung

Publications and source records attributed to Sungkyu Jung.

At least 37 records · Page 2Linked to original sources

arXiv2023

Averaging symmetric positive-definite matrices on the space of eigen-decompositions

We study extensions of Fréchet means for random objects in the space ${\rm Sym}^+(p)$ of $p \times p$ symmetric positive-definite matrices using the scaling-rotation geometric framework introduced by Jung et al. [\textit{SIAM J. Matrix. Anal. Appl.} \textbf{36} (2015) 1180-1201]. The scaling-rotation framework is designed to enjoy a clearer interpretation of the changes in random ellipsoids in terms of scaling and rotation. In this work, we formally define the \emph{scaling-rotation (SR) mean set} to be the set of Fréchet means in ${\rm Sym}^+(p)$ with respect to the scaling-rotation distance. Since computing such means requires a difficult optimization, we also define the \emph{partial scaling-rotation (PSR) mean set} lying on the space of eigen-decompositions as a proxy for the SR mean set. The PSR mean set is easier to compute and its projection to ${\rm Sym}^+(p)$ often coincides with SR mean set. Minimal conditions are required to ensure that the mean sets are non-empty. Because eigen-decompositions are never unique, neither are PSR means, but we give sufficient conditions for the sample PSR mean to be unique up to the action of a certain finite group. We also establish strong consistency of the sample PSR means as estimators of the population PSR mean set, and a central limit theorem. In an application to multivariate tensor-based morphometry, we demonstrate that a two-group test using the proposed PSR means can have greater power than the two-group test using the usual affine-invariant geometric framework for symmetric positive-definite matrices.

stat.ME↗

arXiv2023

Differentially Private Multivariate Statistics with an Application to Contingency Table Analysis

Differential privacy (DP) has become a rigorous central concept for privacy protection in the past decade. We use Gaussian differential privacy (GDP) in gauging the level of privacy protection for releasing statistical summaries from data. The GDP is a natural and easy-to-interpret differential privacy criterion based on the statistical hypothesis testing framework. The Gaussian mechanism is a natural and fundamental mechanism that can be used to perturb multivariate statistics to satisfy a $μ$-GDP criterion, where $μ>0$ stands for the level of privacy protection. Requiring a certain level of differential privacy inevitably leads to a loss of statistical utility. We improve ordinary Gaussian mechanisms by developing rank-deficient James-Stein Gaussian mechanisms for releasing private multivariate statistics, and show that the proposed mechanisms have higher statistical utilities. Laplace mechanisms, the most commonly used mechanisms in the pure DP framework, are also investigated under the GDP criterion. We show that optimal calibration of multivariate Laplace mechanisms requires more information on the statistic than just the global sensitivity, and derive the minimal amount of Laplace perturbation for releasing $μ$-GDP contingency tables. Gaussian mechanisms are shown to have higher statistical utilities than Laplace mechanisms, except for very low levels of privacy. The utility of proposed multivariate mechanisms is further demonstrated using differentially private hypotheses tests on contingency tables. Bootstrap-based goodness-of-fit and homogeneity tests, utilizing the proposed rank-deficient James--Stein mechanisms, exhibit higher powers than natural competitors.

stat.ME↗

arXiv2023

Integrative decomposition of multi-source data by identifying partially-joint score subspaces

Analysis of multi-source dataset, where data on the same objects are collected from multiple sources, is of rising importance in many fields, most notably in multi-omics biology. A novel framework and algorithms for integrative decomposition of such multi-source data are proposed to identify and sort out common factor scores in terms of whether the scores are relevant to all data sources (fully joint), to some data sources (partially joint), or to a single data source. The key difference between the proposed method and existing approaches is that raw source-wise factor score subspaces are utilized in the identification of the partially-joint block-wise association structure. To identify common score subspaces, which may be partially joint to some of data sources, from noisy observations, the proposed algorithm sequentially computes one-dimensional flag means among source-wise score subspaces, then collects the subspaces that are close to the mean. The proposed decomposition boasts fast computational speed, and is superior in identifying the true partially-joint association structure and recovering the joint loading and score subspaces than competing approaches. The proposed decomposition is applied to a blood cancer multi-omics data set, containing measurements from three data sources. Our method identifies a latent score, partially joint to the drug panel and methylation profile data sources but not relevant to RNA sequencing profiles, which helps discovering hidden clusters in the data.

stat.ME↗

arXiv2020

Adjusting systematic bias in high dimensional principal component scores

Principal component analysis continues to be a powerful tool in dimension reduction of high dimensional data. We assume a variance-diverging model and use the high-dimension, low-sample-size asymptotics to show that even though the principal component directions are not consistent, the sample and prediction principal component scores can be useful in revealing the population structure. We further show that these scores are biased, and the bias is asymptotically decomposed into rotation and scaling parts. We propose methods of bias-adjustment that are shown to be consistent and work well in the high dimensional situations with small sample sizes. The potential advantage of bias-adjustment is demonstrated in a classification setting.

math.ST↗

arXiv2020

Covariance-engaged Classification of Sets via Linear Programming

Set classification aims to classify a set of observations as a whole, as opposed to classifying individual observations separately. To formally understand the unfamiliar concept of binary set classification, we first investigate the optimal decision rule under the normal distribution, which utilizes the empirical covariance of the set to be classified. We show that the number of observations in the set plays a critical role in bounding the Bayes risk. Under this framework, we further propose new methods of set classification. For the case where only a few parameters of the model drive the difference between two classes, we propose a computationally-efficient approach to parameter estimation using linear programming, leading to the Covariance-engaged LInear Programming Set (CLIPS) classifier. Its theoretical properties are investigated for both independent case and various (short-range and long-range dependent) time series structures among observations within each set. The convergence rates of estimation errors and risk of the CLIPS classifier are established to show that having multiple observations in a set leads to faster convergence rates, compared to the standard classification situation in which there is only one observation in the set. The applicable domains in which the CLIPS performs better than competitors are highlighted in a comprehensive simulation study. Finally, we illustrate the usefulness of the proposed methods in classification of real image data in histopathology.

stat.ML↗

arXiv2018

Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem

We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEP). The GEP arises in a number of modern data-analytic situations and statistical methods, including principal component analysis (PCA), multiclass linear discriminant analysis (LDA), canonical correlation analysis (CCA), sufficient dimension reduction (SDR) and invariant co-ordinate selection. We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides accurate estimation of the true eigenspace, when it is sparse. Also proposed is a computationally more efficient alternative, which works well for PCA and LDA problems. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We demonstrate applications of the proposed algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA and SDR. Supplementary materials are available online.

stat.ME↗

arXiv2018

Continuum directions for supervised dimension reduction

Dimension reduction of multivariate data supervised by auxiliary information is considered. A series of basis for dimension reduction is obtained as minimizers of a novel criterion. The proposed method is akin to continuum regression, and the resulting basis is called continuum directions. With a presence of binary supervision data, these directions continuously bridge the principal component, mean difference and linear discriminant directions, thus ranging from unsupervised to fully supervised dimension reduction. High-dimensional asymptotic studies of continuum directions for binary supervision reveal several interesting facts. The conditions under which the sample continuum directions are inconsistent, but their classification performance is good, are specified. While the proposed method can be directly used for binary and multi-category classification, its generalizations to incorporate any form of auxiliary data are also presented. The proposed method enjoys fast computation, and the performance is better or on par with more computer-intensive alternatives.

stat.ME↗

arXiv2017

Uniqueness questions in a scaling-rotation geometry on the space of symmetric positive-definite matrices

Jung et al. (2015) introduced a geometric structure on ${\rm Sym}^+(p)$, the set of $p \times p$ symmetric positive-definite matrices, based on eigen-decomposition. Eigenstructure determines both a stratification of ${\rm Sym}^+(p)$, defined by eigenvalue multiplicities, and fibers of the "eigen-composition" map $F:M(p):=SO(p)\times{\rm Diag}^+(p)\to{\rm Sym}^+(p)$. When $M(p)$ is equipped with a suitable Riemannian metric, the fiber structure leads to notions of scaling-rotation distance between $X,Y\in {\rm Sym}^+(p)$, the distance in $M(p)$ between fibers $F^{-1}(X)$ and $F^{-1}(Y)$, and minimal smooth scaling-rotation (MSSR) curves, images in ${\rm Sym}^+(p)$ of minimal-length geodesics connecting two fibers. In this paper we study the geometry of the triple $(M(p),F,{\rm Sym}^+(p))$, focusing on some basic questions: For which $X,Y$ is there a unique MSSR curve from $X$ to $Y$? More generally, what is the set ${\cal M}(X,Y)$ of MSSR curves from $X$ to $Y$? This set is influenced by two potential types of non-uniqueness. We translate the question of whether the second type can occur into a question about the geometry of Grassmannians $G_m({\bf R}^p)$, with $m$ even, that we answer for $p\leq 4$ and $p\geq 11$. Our method of proof also yields an interesting half-angle formula concerning principal angles between subspaces of ${\bf R}^p$ whose dimensions may or may not be equal. The general-$p$ results concerning MSSR curves and scaling-rotation distance that we establish here underpin the explicit $p=3$ results in Groisser et al. (2017). Addressing the uniqueness-related questions requires a thorough understanding of the fiber structure of $M(p)$, which we also provide.

math.MG↗

arXiv2017

On the number of principal components in high dimensions

We consider the problem of how many components to retain in the application of principal component analysis when the dimension is much higher than the number of observations. To estimate the number of components, we propose to sequentially test skewness of the squared lengths of residual scores that are obtained by removing leading principal components. The residual lengths are asymptotically left-skewed if all principal components with diverging variances are removed, and right-skewed if not. The proposed estimator is shown to be consistent, performs well in high-dimensional simulation studies, and provides reasonable estimates in a number of real data examples.

stat.ME↗

arXiv2017

Combined Analysis of Amplitude and Phase Variations in Functional Data

When functional data manifest amplitude and phase variations, a commonly-employed framework for analyzing them is to take away the phase variation through a function alignment and then to apply standard tools to the aligned functions. A downside of this approach is that the important variations contained in the phases are completely ignored. To combine both of amplitude and phase variations, we propose a variant of principal component analysis (PCA) that captures non-linear components representing the amplitude, phase and their associations simultaneously. The proposed method, which we call functional combined PCA, is aimed to provide more efficient dimension reduction with interpretable components, in particular when the amplitudes and phases are clearly associated. We model principal components by non-linearly combining time-warping functions and aligned functions. A data-adaptive weighting procedure helps our dimension reduction to attain a maximal explaining power of observed functions. We also discuss an application of functional canonical correlation analysis in investigation of the correlation structure between the two variations. We show that for two sets of real data the proposed method provides interpretable major non-linear components, which are not typically found in the usual functional PCA.

stat.ME↗

arXiv2017

Small sphere distributions for directional data with application to medical imaging

We propose new small-sphere distributional families for modeling multivariate directional data on $(\mathbb{S}^{p-1})^K$ for $p \ge 3$ and $K \ge 1$. In a special case of univariate directions in $\Re^3$, the new densities model random directions on $\mathbb{S}^2$ with a tendency to vary along a small circle on the sphere, and with a unique mode on the small circle. The proposed multivariate densities enable us to model association among multivariate directions, and are useful in medical imaging, where multivariate directions are used to represent shape and shape changes of 3-dimensional objects. When the underlying objects are rotationally deformed under noise, for instance, twisted and/or bend, corresponding directions tend to follow the proposed small-sphere distributions. The proposed models have several advantages over other methods analyzing small-circle-concentrated data, including inference procedures on the association and small-circle fitting. We demonstrate the use of the proposed multivariate small-sphere distributions in analyses of skeletally-represented object shapes and human knee gait data.

stat.ME↗

arXiv2017

Incorporating Covariates into Integrated Factor Analysis of Multi-View Data

In modern biomedical research, it is ubiquitous to have multiple data sets measured on the same set of samples from different views (i.e., multi-view data). For example, in genetic studies, multiple genomic data sets at different molecular levels or from different cell types are measured for a common set of individuals to investigate genetic regulation. Integration and reduction of multi-view data have the potential to leverage information in different data sets, and to reduce the magnitude and complexity of data for further statistical analysis and interpretation. In this paper, we develop a novel statistical model, called supervised integrated factor analysis (SIFA), for integrative dimension reduction of multi-view data while incorporating auxiliary covariates. The model decomposes data into joint and individual factors, capturing the joint variation across multiple data sets and the individual variation specific to each set respectively. Moreover, both joint and individual factors are partially informed by auxiliary covariates via nonparametric models. We devise a computationally efficient Expectation-Maximization (EM) algorithm to fit the model under some identifiability conditions. We apply the method to the Genotype-Tissue Expression (GTEx) data, and provide new insights into the variation decomposition of gene expression in multiple tissues. Extensive simulation studies and an additional application to a pediatric growth study demonstrate the advantage of the proposed method over competing methods.

stat.ME↗

arXiv2017

Geometric foundations for scaling-rotation statistics on symmetric positive definite matrices: minimal smooth scaling-rotation curves in low dimensions

We investigate a geometric computational framework, called the "scaling-rotation framework", on ${\rm Sym}^+(p)$, the set of $p \times p$ symmetric positive-definite (SPD) matrices. The purpose of our study is to lay geometric foundations for statistical analysis of SPD matrices, in situations in which eigenstructure is of fundamental importance, for example diffusion-tensor imaging (DTI). Eigen-decomposition, upon which the scaling-rotation framework is based, determines both a stratification of ${\rm Sym}^+(p)$, defined by eigenvalue multiplicities, and fibers of the "eigen-composition" map $SO(p)\times{\rm Diag}^+(p)\to{\rm Sym}^+(p)$. This leads to the notion of scaling-rotation distance [Jung et al. (2015)], a measure of the minimal amount of scaling and rotation needed to transform an SPD matrix, $X,$ into another, $Y,$ by a smooth curve in ${\rm Sym}^+(p)$. Our main goal in this paper is the systematic characterization and analysis of minimal smooth scaling-rotation (MSSR) curves, images in ${\rm Sym}^+(p)$ of minimal-length geodesics connecting two fibers in the "upstairs" space $SO(p)\times{\rm Diag}^+(p)$. The length of such a geodesic connecting the fibers over $X$ and $Y$ is what we define to be the scaling-rotation distance from $X$ to $Y.$ For the important low-dimensional case $p = 3$ (the home of DTI), we find new explicit formulas for MSSR curves and for the scaling-rotation distance, and identify ${\cal M}(X,Y)$ in all "nontrivial" cases. The quaternionic representation of $SO(3)$ is used in these computations. We also provide closed-form expressions for scaling-rotation distance and MSSR curves for the case $p = 2$.

math.MG↗

arXiv2016

Sparse Generalized Eigenvalue Problem with Application to Canonical Correlation Analysis for Integrative Analysis of Methylation and Gene Expression Data

We present a method for individual and integrative analysis of high dimension, low sample size data that capitalizes on the recurring theme in multivariate analysis of projecting higher dimensional data onto a few meaningful directions that are solutions to a generalized eigenvalue problem. We propose a general framework, called SELP (Sparse Estimation with Linear Programming), with which one can obtain a sparse estimate for a solution vector of a generalized eigenvalue problem. We demonstrate the utility of SELP on canonical correlation analysis for an integrative analysis of methylation and gene expression profiles from a breast cancer study, and we identify some genes known to be associated with breast carcinogenesis, which indicates that the proposed method is capable of generating biologically meaningful insights. Simulation studies suggest that the proposed method performs competitive in comparison with some existing methods in identifying true signals in various underlying covariance structures.

stat.ME↗

arXiv2016

Inference on subspheres model for directional data

Modeling deformations of a real object is an important task in computer vision, biomedical engineering and biomechanics. In this paper, we focus on a situation where a three-dimensional object is rotationally deformed about a fixed axis, and assume that many independent observations are available. Such a problem is generalized to an estimation of concentric, co-dimension 1, subspheres of a polysphere. We formulate least-square estimators as generalized Fréchet means, and evaluate the consistency and asymptotic normality.

math.ST↗

arXiv2015

Scaling-rotation distance and interpolation of symmetric positive-definite matrices

We introduce a new geometric framework for the set of symmetric positive-definite (SPD) matrices, aimed to characterize deformations of SPD matrices by individual scaling of eigenvalues and rotation of eigenvectors of the SPD matrices. To characterize the deformation, the eigenvalue-eigenvector decomposition is used to find alternative representations of SPD matrices, and to form a Riemannian manifold so that scaling and rotations of SPD matrices are captured by geodesics on this manifold. The problems of non-unique eigen-decompositions and eigenvalue multiplicities are addressed by finding minimal-length geodesics, which gives rise to a distance and an interpolation method for SPD matrices. Computational procedures to evaluate the minimal scaling--rotation deformations and distances are provided for the most useful cases of $2 \times 2$ and $3 \times 3$ SPD matrices. In the new geometric framework, minimal scaling--rotation curves interpolate eigenvalues at constant logarithmic rate, and eigenvectors at constant angular rate. In the context of diffusion tensor imaging, this results in better behavior of the trace, determinant and fractional anisotropy of interpolated SPD matrices in typical cases.

math.MG↗

arXiv2014

A Statistical Approach to Set Classification by Feature Selection with Applications to Classification of Histopathology Images

Set classification problems arise when classification tasks are based on sets of observations as opposed to individual observations. In set classification, a classification rule is trained with $N$ sets of observations, where each set is labeled with class information, and the prediction of a class label is performed also with a set of observations. Data sets for set classification appear, for example, in diagnostics of disease based on multiple cell nucleus images from a single tissue. Relevant statistical models for set classification are introduced, which motivate a set classification framework based on context-free feature extraction. By understanding a set of observations as an empirical distribution, we employ a data-driven method to choose those features which contain information on location and major variation. In particular, the method of principal component analysis is used to extract the features of major variation. Multidimensional scaling is used to represent features as vector-valued points on which conventional classifiers can be applied. The proposed set classification approaches achieve better classification results than competing methods in a number of simulated data examples. The benefits of our method are demonstrated in an analysis of histopathology images of cell nuclei related to liver cancer.

stat.ME↗

arXiv2011

Principal arc analysis on direct product manifolds

We propose a new approach to analyze data that naturally lie on manifolds. We focus on a special class of manifolds, called direct product manifolds, whose intrinsic dimension could be very high. Our method finds a low-dimensional representation of the manifold that can be used to find and visualize the principal modes of variation of the data, as Principal Component Analysis (PCA) does in linear spaces. The proposed method improves upon earlier manifold extensions of PCA by more concisely capturing important nonlinear modes. For the special case of data on a sphere, variation following nongeodesic arcs is captured in a single mode, compared to the two modes needed by previous methods. Several computational and statistical challenges are resolved. The development on spheres forms the basis of principal arc analysis on more complicated manifolds. The benefits of the method are illustrated by a data example using medial representations in image analysis.

stat.AP↗