arXiv subjects

Yushu Shi

Publications and source records attributed to Yushu Shi.

7 recordsLinked to original sources

arXiv2024

survivalContour: Visualizing predicted survival via colored contour plots

Advances in survival analysis have facilitated unprecedented flexibility in data modeling, yet there remains a lack of tools for graphically illustrating the influence of continuous covariates on predicted survival outcomes. We propose the utilization of a colored contour plot to depict the predicted survival probabilities over time, and provide a Shiny app and R package as implementations of this tool. Our approach is capable of supporting conventional models, including the Cox and Fine-Gray models. However, its capability shines when coupled with cutting-edge machine learning models such as random survival forests and deep neural networks.

stat.AP↗

arXiv2023

The Conditional Prediction Function: A Novel Technique to Control False Discovery Rate for Complex Models

In modern scientific research, the objective is often to identify which variables are associated with an outcome among a large class of potential predictors. This goal can be achieved by selecting variables in a manner that controls the the false discovery rate (FDR), the proportion of irrelevant predictors among the selections. Knockoff filtering is a cutting-edge approach to variable selection that provides FDR control. Existing knockoff statistics frequently employ linear models to assess relationships between features and the response, but the linearity assumption is often violated in real world applications. This may result in poor power to detect truly prognostic variables. We introduce a knockoff statistic based on the conditional prediction function (CPF), which can pair with state-of-art machine learning predictive models, such as deep neural networks. The CPF statistics can capture the nonlinear relationships between predictors and outcomes while also accounting for correlation between features. We illustrate the capability of the CPF statistics to provide superior power over common knockoff statistics with continuous, categorical, and survival outcomes using repeated simulations. Knockoff filtering with the CPF statistics is demonstrated using (1) a residential building dataset to select predictors for the actual sales prices and (2) the TCGA dataset to select genes that are correlated with disease staging in lung cancer patients.

stat.ME↗

arXiv2023

CAT: a conditional association test for microbiome data using a leave-out approach

In microbiome analysis, researchers often seek to identify taxonomic features associated with an outcome of interest. However, microbiome features are intercorrelated and linked by phylogenetic relationships, making it challenging to assess the association between an individual feature and an outcome. Researchers have developed global tests for the association of microbiome profiles with outcomes using beta diversity metrics which offer robustness to extreme values and can incorporate information on the phylogenetic tree structure. Despite the popularity of global association testing, most existing methods for follow-up testing of individual features only consider the marginal effect and do not provide relevant information for the design of microbiome interventions. This paper proposes a novel conditional association test, CAT, which can account for other features and phylogenetic relatedness when testing the association between a feature and an outcome. CAT adopts a leave-out method, measuring the importance of a feature in predicting the outcome by removing that feature from the data and quantifying how much the association with the outcome is weakened through the change in the coefficient of determination. By leveraging global tests including PERMANOVA and MiRKAT-based methods, CAT allows association testing for continuous, binary, categorical, count, survival, and correlated outcomes. Our simulation and real data application results illustrate the potential of CAT to inform the design of microbiome interventions aimed at improving clinical outcomes.

stat.ME↗

arXiv2022

Sparse tree-based clustering of microbiome data to characterize microbiome heterogeneity in pancreatic cancer

There is a keen interest in characterizing variation in the microbiome across cancer patients, given increasing evidence of its important role in determining treatment outcomes. Here our goal is to discover subgroups of patients with similar microbiome profiles. We propose a novel unsupervised clustering approach in the Bayesian framework that innovates over existing model-based clustering approaches, such as the Dirichlet multinomial mixture model, in three key respects: we incorporate feature selection, learn the appropriate number of clusters from the data, and integrate information on the tree structure relating the observed features. We compare the performance of our proposed method to existing methods on simulated data designed to mimic real microbiome data. We then illustrate results obtained for our motivating data set, a clinical study aimed at characterizing the tumor microbiome of pancreatic cancer patients.

stat.AP↗

arXiv2021

Nonparametric Bayesian Knockoff Generators for Feature Selection Under Complex Data Structure

The recent proliferation of high-dimensional data, such as electronic health records and genetics data, offers new opportunities to find novel predictors of outcomes. Presented with a large set of candidate features, interest often lies in selecting the ones most likely to be predictive of an outcome for further study. Controlling the false discovery rate (FDR) at a specified level is often desired in evaluating these variables. Knockoff filtering is an innovative strategy for conducting FDR-controlled feature selection. This paper proposes a nonparametric Bayesian model for generating high-quality knockoff copies that can improve the accuracy of predictive feature identification for variables arising from complex distributions, which can be skewed, highly dispersed and/or a mixture of distributions. This paper provides a detailed description for generating knockoff copies from a GDPM model via MCMC posterior sampling. Additionally, we provide a theoretical guarantee on the robustness of the knockoff procedure. Through simulations, the method is shown to identify important features with accurate FDR control and improved power over the popular second-order Gaussian knockoff generator. Furthermore, the model is compared with finite Gaussian mixture knockoff generator in FDR and power. The proposed technique is applied for detecting genes predictive of survival in ovarian cancer patients using data from The Cancer Genome Atlas (TCGA).

stat.ME↗

arXiv2020

ProgPermute: Progressive permutation for a dynamic representation of the robustness of microbiome discoveries

Identification of features is a critical task in microbiome studies that is complicated by the fact that microbial data are high dimensional and heterogeneous. Masked by the complexity of the data, the problem of separating signals from noise becomes challenging and troublesome. For instance, when performing differential abundance tests, multiple testing adjustments tend to be overconservative, as the probability of a type I error (false positive) increases dramatically with the large numbers of hypotheses. Moreover, the grouping effect of interest can be obscured by heterogeneity. These factors can incorrectly lead to the conclusion that there are no differences in the microbiome compositions. We translate and represent the problem of identifying differential features as a dynamic layout of separating the signal from its random background. We propose progressive permutation as a method to achieve this process and show converging patterns. More specifically, we progressively permute the grouping factor labels of the microbiome samples and perform multiple differential abundance tests in each scenario. We then compare the signal strength of the top features from the original data with their performance in permutations, and observe an apparent decreasing trend if these top features are true positives identified from the data. We have developed this into a user-friendly RShiny tool and R package, which consist of functions that can convey the overall association between the microbiome and the grouping factor, rank the robustness of the discovered microbes, and list the discoveries, their effect sizes, and individual abundances.

stat.ME↗

arXiv2020

aPCoA: Covariate Adjusted Principal Coordinates Analysis

In fields such as ecology, microbiology, and genomics, non-Euclidean distances are widely applied to describe pairwise dissimilarity between samples. Given these pairwise distances, principal coordinates analysis (PCoA) is commonly used to construct a visualization of the data. However, confounding covariates can make patterns related to the scientific question of interest difficult to observe. We provide aPCoA as an easy-to-use tool, available as both an R package and a Shiny app, to improve data visualization in this context, enabling enhanced presentation of the effects of interest.

q-bio.QM↗