arXiv subjects

Chandler Zuo

Publications and source records attributed to Chandler Zuo.

3 recordsLinked to original sources

arXiv2018

Regularization Effect of Fast Gradient Sign Method and its Generalization

Fast Gradient Sign Method (FGSM) is a popular method to generate adversarial examples that make neural network models robust against perturbations. Despite its empirical success, its theoretical property is not well understood. This paper develops theory to explain the regularization effect of Generalized FGSM, a class of methods to generate adversarial examples. Motivated from the relationship between FGSM and LASSO penalty, the asymptotic properties of Generalized FGSM are derived in the Generalized Linear Model setting, which is essentially the 1-layer neural network setting with certain activation functions. In such simple neural network models, I prove that Generalized FGSM estimation is root n-consistent and weakly oracle under proper conditions. The asymptotic results are also highly similar to penalized likelihood estimation. Nevertheless, Generalized FGSM introduces additional bias when data sampling is not sign neutral, a concept I introduce to describe the balance-ness of the noise signs. Although the theory in this paper is developed under simple neural network settings, I argue that it may give insights and justification for FGSM in deep neural network settings as well.

stat.ML↗

arXiv2017

Calibration for Stratified Classification Models

In classification problems, sampling bias between training data and testing data is critical to the ranking performance of classification scores. Such bias can be both unintentionally introduced by data collection and intentionally introduced by the algorithm, such as under-sampling or weighting techniques applied to imbalanced data. When such sampling bias exists, using the raw classification score to rank observations in the testing data can lead to suboptimal results. In this paper, I investigate the optimal calibration strategy in general settings, and develop a practical solution for one specific sampling bias case, where the sampling bias is introduced by stratified sampling. The optimal solution is developed by analytically solving the problem of optimizing the ROC curve. For practical data, I propose a ranking algorithm for general classification models with stratified data. Numerical experiments demonstrate that the proposed algorithm effectively addresses the stratified sampling bias issue. Interestingly, the proposed method shows its potential applicability in two other machine learning areas: unsupervised learning and model ensembling, which can be future research topics.

stat.ME↗

arXiv2016

A Hierarchical Framework for State Space Matrix Inference and Clustering

In recent years, a large number of genomic and epigenomic studies have been focusing on the integrative analysis of multiple experimental datasets measured over a large number of observational units. The objectives of such studies include not only inferring a hidden state of activity for each unit over individual experiments, but also detecting highly associated clusters of units based on their inferred states. In this paper, we develop the MBASIC (Matrix Based Analysis for State-space Inference and Clustering) framework. MBASIC consists of two parts: state-space mapping and state-space clustering. In state-space mapping, it maps observations onto a finite state-space, representing the activation states of units across conditions. In state-space clustering, MBASIC incorporates a finite mixture model to cluster the units based on their inferred state-space profiles across all conditions. Both the state-space mapping and clustering can be simultaneously estimated through an Expectation-Maximization algorithm. MBASIC flexibly adapts to a large number of parametric distributions for the observed data, as well as the heterogeneity in replicate experiments. In our data-driven simulation studies, MBASIC showed significant accuracy in recovering both the underlying state-space variables and clustering structures. We applied MBASIC to two genome research problems using large numbers of datasets from the ENCODE project. In both studies, MBASIC showed higher levels of raw data fidelity than analyzing these data with a two-step approach using ENCODE results on transcription factor occupancy data.

stat.ME↗