arXiv subjects

Kwun Chuen Gary Chan

Publications and source records attributed to Kwun Chuen Gary Chan.

18 recordsLinked to original sources

arXiv2026

Two-stage Adaptive Testing of Large-scale Mediation Hypotheses

In testing large-scale mediation hypotheses, the exposure-mediator and mediator-outcome path-specific p-values obtained for each hypothesis from the same sample can be combined into a pair of asymptotically independent p-values, which can then be used to test a global null hypothesis and a composite mediation null hypothesis, respectively. A first-stage screening procedure targeting the more stringent global null can effectively reduce the number of mediation hypotheses to be tested in the second stage, which in turn reduces the conservativeness of multiple comparison. The framework can incorporate any data-adaptive choice of screening threshold in Stage 1, and any step-up false discovery rate control method in Stage 2. The proposed procedure controls the false discovery rate asymptotically while being consistently well powered across a wide range of scenarios.

stat.ME↗

arXiv2026

Likelihood-based Spacings Goodness-of-Fit Statistics for Univariate Shape-constrained Densities

A variety of statistics based on sample spacings have been studied for testing goodness-of-fit to parametric distributions. To test the goodness-of-fit to a nonparametric class of univariate shape-constrained densities, including widely studied classes such as k-monotone and log-concave densities, a likelihood ratio test with a working alternative density estimate based on the spacings of the observations is considered, and is shown to be asymptotically normal and distribution-free under the null, consistent under fixed and certain local alternatives, and admits bootstrap calibration. The distribution-freeness under the null comes from the fact that the asymptotic dominant term depends only on a function of the spacings of transformed outcomes that are uniformly distributed. Applications and extensions of theoretical results in the literature of shape-constrained estimation are required to show that the average log-density ratio converges to zero at a faster rate than the sample spacing term under the null, and diverges under the alternatives. Numerical studies are conducted to demonstrate that the test is applicable to various classes of shape-constrained densities and has a good balance between type-I error control under the null and power under alternative distributions.

math.ST↗

arXiv2026

Dimension-reduced outcome-weighted learning for estimating individualized treatment regimes in observational studies

Individualized treatment regimes (ITRs) aim to improve clinical outcomes by assigning treatment based on patient-specific characteristics. However, existing methods often struggle with high-dimensional covariates, limiting accuracy, interpretability, and real-world applicability. We propose a novel sufficient dimension reduction approach that directly targets the contrast between potential outcomes and identifies a low-dimensional subspace of the covariates capturing treatment effect heterogeneity. This reduced representation enables more accurate estimation of optimal ITRs through outcome-weighted learning. To accommodate observational data, our method incorporates kernel-based covariate balancing, allowing treatment assignment to depend on the full covariate set and avoiding the restrictive assumption that the subspace sufficient for modeling heterogeneous treatment effects is also sufficient for confounding adjustment. We show that the proposed method achieves universal consistency, i.e., its risk converges to the Bayes risk, under mild regularity conditions. We demonstrate its finite sample performance through simulations and an analysis of intensive care unit sepsis patient data to determine who should receive transthoracic echocardiography.

stat.ML↗

arXiv2025

Factors affecting power in stepped wedge trials when the treatment effect varies with time

Stepped wedge cluster randomized trials (SW-CRTs) have historically been analyzed using immediate treatment (IT) models, which assume the effect of the treatment is immediate after treatment initiation and subsequently remains constant over time. However, recent research has shown that this assumption can lead to severely misleading results if treatment effects vary with exposure time, i.e. time since the intervention started. Models that account for time-varying treatment effects, such as the exposure time indicator (ETI) model, allow researchers to target estimands such as the time-averaged treatment effect (TATE) over an interval of exposure time, or the point treatment effect (PTE) representing a treatment contrast at one time point. However, this increased flexibility results in reduced power. In this paper, we use public power calculation software and simulation to characterize factors affecting SW-CRT power. Key elements include choice of estimand, study design considerations, and analysis model selection. or common SW-CRT designs, the sample size (clusters per sequence or individuals per cluster-period) must be increased substantially, commonly by a factor of 1.5 to 3, but often by much more, to maintain 90\% power when switching from an IT model to an ETI model (targeting the TATE over the study). However, the inflation factor is lower for TATE estimands over shorter periods that exclude longer exposure times. In general, SW-CRT designs (including the "staircase" variant) have much greater power for estimating "short-term effects" relative to "long-term effects". For an ETI model targeting a TATE estimand, substantial power can be gained by adding time points to the start of the study or increasing baseline sample size, but surprisingly little power is gained from adding time points to the end of the study. More restrictive choices for modeling the exposure... [truncated]

stat.ME↗

arXiv2025

GMM with Many Weak Moment Conditions and Nuisance Parameters: General Theory and Applications to Causal Inference

Weak identification arises in many statistical problems when key variables exhibit weak correlations-for example, when instrumental variables correlate weakly with treatment, or when proxy variables correlate weakly with unmeasured confounders. Under weak identification, standard estimation methods such as the generalized method of moments (GMM) can produce substantial bias, both in finite samples and asymptotically. This challenge is compounded in modern applications that require estimating many nuisance parameters. This paper develops a framework for estimation and inference of a finite-dimensional target parameter in general moment models with the number of weak moment conditions and nuisance parameters growing with sample size. We analyze a general two-step debiasing estimator that accommodates flexible, possibly nonparametric first-step estimation of nuisance parameters, in which Neyman orthogonality plays a more critical role in obtaining debiased inference than in conventional settings with strong identification. Under a many-weak-moment asymptotic regime, we establish the estimator's consistency and asymptotic normality. We provide high-level conditions for the general setting and demonstrate their application to two important special cases: inference with weak instruments and inference with weak proxies.

math.ST↗

arXiv2025

Weighted Brier Score -- an Overall Summary Measure for Risk Prediction Models with Clinical Utility Consideration

As advancements in novel biomarker-based algorithms and models accelerate disease risk prediction and stratification in medicine, it is crucial to evaluate these models within the context of their intended clinical application. Prediction models output the absolute risk of disease; subsequently, patient counseling and shared decision-making are based on the estimated individual risk and cost-benefit assessment. The overall impact of the application is often referred to as clinical utility, which received significant attention in terms of model assessment lately. The classic Brier score is a popular measure of prediction accuracy; however, it is insufficient for effectively assessing clinical utility. To address this limitation, we propose a class of weighted Brier scores that aligns with the decision-theoretic framework of clinical utility. Additionally, we decompose the weighted Brier score into discrimination and calibration components, examining how weighting influences the overall score and its individual components. Through this decomposition, we link the weighted Brier score to the $H$ measure, which has been proposed as a coherent alternative to the area under the receiver operating characteristic curve. This theoretical link to the $H$ measure further supports our weighting method and underscores the essential elements of discrimination and calibration in risk prediction evaluation. The practical use of the weighted Brier score as an overall summary is demonstrated using data from the Prostate Cancer Active Surveillance Study (PASS).

stat.ME↗

arXiv2025

Efficient Estimation under Multiple Missing Patterns via Balancing Weights

As one of the most commonly seen data challenges, missing data, in particular, multiple, non-monotone missing patterns, complicates estimation and inference due to the fact that missingness mechanisms are often not missing at random, and conventional methods cannot be applied. Pattern graphs have recently been proposed as a tool to systematically relate various observed patterns in the sample. We extend its scope to the estimation of parameters defined by moment equations, including common regression models, via solving weighted estimating equations with weights constructed using a sequential balancing approach. These novel weights are carefully crafted to address the instability issue of the straightforward approach based on local balancing. We derive the efficiency bound for the model parameters and show that our proposed method, albeit relatively simple, is asymptotically efficient. Simulation results demonstrate the superior performance of the proposed method, and real-data applications illustrate how the results are robust to the choice of identification assumptions.

stat.ME↗

arXiv2024

Balancing Weights for Non-monotone Missing Data

Balancing weights have been widely applied to single or monotone missingness due to empirical advantages over likelihood-based methods and inverse probability weighting approaches. This paper considers non-monotone missing data under the complete-case missing variable condition (CCMV), a case of missing not at random (MNAR). Using relationships between each missing pattern and the complete-case subsample, we construct a weighted estimator for estimation, where the weight is a sum of ratios of the conditional probability of observing a particular missing pattern versus that of observing the complete-case, given the variables observed in the corresponding missing pattern. However, plug-in estimators of the propensity odds can be unbounded and lead to unstable estimation. Using further relations between propensity odds and balancing of moments across response patterns, we employ tailored loss functions, each encouraging empirical balance across patterns to estimate propensity odds flexibly using a functional basis expansion. We propose two penalizations to control propensity odds model smoothness and empirical imbalance. We study the asymptotic properties of the proposed estimators and show that they are consistent under mild smoothness assumptions. Asymptotic normality and efficiency are developed. Simulation results show the superior performance of the proposed method.

stat.ME↗

arXiv2024

Flexible Functional Treatment Effect Estimation

We study treatment effect estimation with functional treatments where the average potential outcome functional is a function of functions, in contrast to continuous treatment effect estimation where the target is a function of real numbers. By considering a flexible scalar-on-function marginal structural model, a weight-modified kernel ridge regression (WMKRR) is adopted for estimation. The weights are constructed by directly minimizing the uniform balancing error resulting from a decomposition of the WMKRR estimator, instead of being estimated under a particular treatment selection model. Despite the complex structure of the uniform balancing error derived under WMKRR, finite-dimensional convex algorithms can be applied to efficiently solve for the proposed weights thanks to a representer theorem. The optimal convergence rate is shown to be attainable by the proposed WMKRR estimator without any smoothness assumption on the true weight function. Corresponding empirical performance is demonstrated by a simulation study and a real data application.

stat.ME↗

arXiv2023

An Effective Multivariate Normality Test via Hessians of Empirical Cumulant Generating Functions

In this article, we propose a new class of consistent tests for $p$-variate normality. These tests are based on the characterization of the standard multivariate normal distribution, that the Hessian of the corresponding cumulant generating function is identical to the $p\times p$ identity matrix and the idea of decomposing the information from the joint distribution into the dependence copula and all marginal distributions. Under the null hypothesis of multivariate normality, our proposed test statistic is independent of the unknown mean vector and covariance matrix so that the distribution-free critical value of the test can be obtained by Monte Carlo simulation. We also derive the asymptotic null distribution of proposed test statistic and establish the consistency of the test against different fixed alternatives. Last but not least, a comprehensive and extensive Monte Carlo study also illustrates that our test is a superb yet computationally convenient competitor to many well-known existing test statistics.

stat.ME↗

arXiv2021

Data Harmonization Via Regularized Nonparametric Mixing Distribution Estimation

Data harmonization is the process by which an equivalence is developed between two variables measuring a common trait. Our problem is motivated by dementia research in which multiple tests are used in practice to measure the same underlying cognitive ability such as language or memory. We connect this statistical problem to mixing distribution estimation. We introduce and study a non-parametric latent trait model, develop a method which enforces uniqueness of the regularized maximum likelihood estimator, show how a nonparametric EM algorithm will converge weakly to its maximizer, and additionally propose a faster algorithm for learning a discretized approximation of the latent distribution. Furthermore, we develop methods to assess goodness of fit for the mixing likelihood which is an area neglected in most mixing distribution estimation problems. We apply our method to the National Alzheimer's Coordination Center Uniform Data Set and show that we can use our method to convert between score measurements and account for the measurement error. We show that this method outperforms standard techniques commonly used in dementia research. Full code is available at https://github.com/SteveJWR/Data-Harmonization-Nonparametric.

stat.ME↗

arXiv2021

Matrix Completion with Model-free Weighting

In this paper, we propose a novel method for matrix completion under general non-uniform missing structures. By controlling an upper bound of a novel balancing error, we construct weights that can actively adjust for the non-uniformity in the empirical risk without explicitly modeling the observation probabilities, and can be computed efficiently via convex optimization. The recovered matrix based on the proposed weighted empirical risk enjoys appealing theoretical guarantees. In particular, the proposed method achieves a stronger guarantee than existing work in terms of the scaling with respect to the observation probabilities, under asymptotically heterogeneous missing settings (where entry-wise observation probabilities can be of different orders). These settings can be regarded as a better theoretical model of missing patterns with highly varying probabilities. We also provide a new minimax lower bound under a class of heterogeneous settings. Numerical experiments are also provided to demonstrate the effectiveness of the proposed method.

stat.ML↗

arXiv2021

Estimation of Partially Conditional Average Treatment Effect by Hybrid Kernel-covariate Balancing

We study nonparametric estimation for the partially conditional average treatment effect, defined as the treatment effect function over an interested subset of confounders. We propose a hybrid kernel weighting estimator where the weights aim to control the balancing error of any function of the confounders from a reproducing kernel Hilbert space after kernel smoothing over the subset of interested variables. In addition, we present an augmented version of our estimator which can incorporate estimations of outcome mean functions. Based on the representer theorem, gradient-based algorithms can be applied for solving the corresponding infinite-dimensional optimization problem. Asymptotic properties are studied without any smoothness assumptions for propensity score function or the need of data splitting, relaxing certain existing stringent assumptions. The numerical performance of the proposed estimator is demonstrated by a simulation study and an application to the effect of a mother's smoking on a baby's birth weight conditioned on the mother's age.

stat.ME↗

arXiv2021

Defining and Estimating Subgroup Mediation Effects with Semi-Competing Risks Data

In many medical studies, an ultimate failure event such as death is likely to be affected by the occurrence and timing of other intermediate clinical events. Both event times are subject to censoring by loss-to-follow-up but the nonterminal event may further be censored by the occurrence of the primary outcome, but not vice versa. To study the effect of an intervention on both events, the intermediate event may be viewed as a mediator, but conventional definition of direct and indirect effects is not applicable due to semi-competing risks data structure. We define three principal strata based on whether the potential intermediate event occurs before the potential failure event, which allow proper definition of direct and indirect effects in one stratum whereas total effects are defined for all strata. We discuss the identification conditions for stratum-specific effects, and proposed a semiparametric estimator based on a multivariate logistic stratum membership model and within-stratum proportional hazards models for the event times. By treating the unobserved stratum membership as a latent variable, we propose an EM algorithm for computation. We study the asymptotic properties of the estimators by the modern empirical process theory and examine the performance of the estimators in numerical studies.

stat.ME↗

arXiv2020

Controlling the False Discovery Rate for Binary Feature Selection via Knockoff

Variable selection has been widely used in data analysis for the past decades, and it becomes increasingly important in the Big Data era as there are usually hundreds of variables available in a dataset. To enhance interpretability of a model, identifying potentially relevant features is often a step before fitting all the features into a regression model. A good variable selection method should effectively control the fraction of false discoveries and ensure large enough power of its selection set. In a lot of contemporary data applications, a great portion of features are coded as binary variables. Binary features are widespread in many fields, from online controlled experiments to genome science to physical statistics. Although there has recently been a handful of literature for provable false discovery rate (FDR) control in variable selection, most of the theoretical analyses were based on some strong dependency assumption or Gaussian assumption among features. In this paper we propose a variable selection method in regression framework for selecting binary features. Under mild conditions, we show that FDR is controlled exactly under a target level in a finite sample if the underlying distribution of the binary features is known. We show in simulations that FDR control is still attained when feature distribution is estimated from data. We also provide theoretical results on the power of our variables selection method in a linear regression model or a logistic regression model. In the restricted settings where competitors exist, we show in simulations and real data application on a HIV antiretroviral therapy dataset that our method has higher power than the competitor.

stat.ME↗

arXiv2014

Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem

In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weighting estimators and the methods can be applied to missing data analysis. Model-based calibration has been proposed in the survey sampling literature, where multidimensional auxiliary variables are first summarized into a predictor function from a working regression model. Usually, one working model is being proposed for each parameter of interest and results in different sets of calibration weights for estimating different parameters. This paper considers calibration using multiple working regression models for estimating a single or multiple parameters. Contrary to a common belief that overfitting hurts efficiency, we present three rather unexpected results. First, when the missing probability is correctly specified and multiple working regression models for the conditional mean are posited, calibration enjoys an oracle property: the same semiparametric efficiency bound is attained as if the true outcome model is known in advance. Second, when the missing data mechanism is misspecified, calibration can still be a consistent estimator when any one of the outcome regression models is correctly specified. Third, a common set of calibration weights can be used to improve efficiency in estimating multiple parameters of interest and can simultaneously attain semiparametric efficiency bounds for all parameters of interest. We provide connections of a wide class of calibration estimators, constructed based on generalized empirical likelihood, to many existing estimators in biostatistics, econometrics and survey sampling and perform simulation studies to show that the finite sample properties of calibration estimators conform well with the theoretical results being studied.

stat.ME↗

arXiv2014

Marginalizable conditional model for clustered ordinal data

We introduce a flexible parametric mixed effects model for correlated binary data, with parameters that can be directly interpreted as marginal odds ratios. This leads to a robust estimation equation with an optimal weighting matrix being the inverse of a genuine model-based covariance matrix. Flexible correlation structures can be imposed by correlated random effects, and correlation parameters can be estimated by solving a composite likelihood score function. Marginal parameters are consistently estimated even when the conditional parametric model is misspecified, and the robust estimation procedure has low estimation efficiency loss compared to the maximum likelihood estimation under a correct model specification. Simulations, analyses of the Madras longitudinal schizophrenia study and British social attributes panel survey were carried out to demonstrate our method.

stat.ME↗

arXiv2010

Backward estimation of stochastic processes with failure events as time origins

Stochastic processes often exhibit sudden systematic changes in pattern a short time before certain failure events. Examples include increase in medical costs before death and decrease in CD4 counts before AIDS diagnosis. To study such terminal behavior of stochastic processes, a natural and direct way is to align the processes using failure events as time origins. This paper studies backward stochastic processes counting time backward from failure events, and proposes one-sample nonparametric estimation of the mean of backward processes when follow-up is subject to left truncation and right censoring. We will discuss benefits of including prevalent cohort data to enlarge the identifiable region and large sample properties of the proposed estimator with related extensions. A SEER--Medicare linked data set is used to illustrate the proposed methodologies.

stat.AP↗