arXiv subjects

Zhiqiang Tan

Publications and source records attributed to Zhiqiang Tan.

At least 37 records · Page 2Linked to original sources

arXiv2022

Model-assisted sensitivity analysis for treatment effects under unmeasured confounding via regularized calibrated estimation

Consider sensitivity analysis for estimating average treatment effects under unmeasured confounding, assumed to satisfy a marginal sensitivity model. At the population level, we provide new representations for the sharp population bounds and doubly robust estimating functions, recently derived by Dorn, Guo, and Kallus. We also derive new, relaxed population bounds, depending on weighted linear outcome quantile regression. At the sample level, we develop new methods and theory for obtaining not only doubly robust point estimators for the relaxed population bounds with respect to misspecification of a propensity score model or an outcome mean regression model, but also model-assisted confidence intervals which are valid if the propensity score model is correctly specified, but the outcome quantile and mean regression models may be misspecified. The relaxed population bounds reduce to the sharp bounds if outcome quantile regression is correctly specified. For a linear outcome mean regression model, the confidence intervals are also doubly robust. Our methods involve regularized calibrated estimation, with Lasso penalties but carefully chosen loss functions, for fitting propensity score and outcome mean and quantile regression models. We present a simulation study and an empirical application to an observational study on the effects of right heart catheterization.

stat.ME↗

arXiv2022

Tractable and Near-Optimal Adversarial Algorithms for Robust Estimation in Contaminated Gaussian Models

Consider the problem of simultaneous estimation of location and variance matrix under Huber's contaminated Gaussian model. First, we study minimum $f$-divergence estimation at the population level, corresponding to a generative adversarial method with a nonparametric discriminator and establish conditions on $f$-divergences which lead to robust estimation, similarly to robustness of minimum distance estimation. More importantly, we develop tractable adversarial algorithms with simple spline discriminators, which can be implemented via nested optimization such that the discriminator parameters can be fully updated by maximizing a concave objective function given the current generator. The proposed methods are shown to achieve minimax optimal rates or near-optimal rates depending on the $f$-divergence and the penalty used. This is the first time such near-optimal error rates are established for adversarial algorithms with linear discriminators under Huber's contamination model. We present simulation studies to demonstrate advantages of the proposed methods over classic robust estimators, pairwise methods, and a generative adversarial method with neural network discriminators.

math.ST↗

arXiv2022

Imputation Maximization Stochastic Approximation with Application to Generalized Linear Mixed Models

Generalized linear mixed models are useful in studying hierarchical data with possibly non-Gaussian responses. However, the intractability of likelihood functions poses challenges for estimation. We develop a new method suitable for this problem, called imputation maximization stochastic approximation (IMSA). For each iteration, IMSA first imputes latent variables/random effects, then maximizes over the complete data likelihood, and finally moves the estimate towards the new maximizer while preserving a proportion of the previous value. The limiting point of IMSA satisfies a self-consistency property and can be less biased in finite samples than the maximum likelihood estimator solved by score-equation based stochastic approximation (ScoreSA). Numerically, IMSA can also be advantageous over ScoreSA in achieving more stable convergence and respecting the parameter ranges under various transformations such as nonnegative variance components. This is corroborated through our simulation studies where IMSA consistently outperforms ScoreSA.

stat.ME↗

arXiv2022

High-dimensional model-assisted inference for treatment effects with multi-valued treatments

Consider estimation of average treatment effects with multi-valued treatments using augmented inverse probability weighted (IPW) estimators, depending on outcome regression and propensity score models in high-dimensional settings. These regression models are often fitted by regularized likelihood-based estimation, while ignoring how the fitted functions are used in the subsequent inference about the treatment parameters. Such separate estimation can be associated with known difficulties in existing methods. We develop regularized calibrated estimation for fitting propensity score and outcome regression models, where sparsity-including penalties are employed to facilitate variable selection but the loss functions are carefully chosen such that valid confidence intervals can be obtained under possible model misspecification. Unlike in the case of binary treatments, the usual augmented IPW estimator is generalized by allowing different copies of coefficient estimators in outcome regression to ensure just-identification. For propensity score estimation, the new loss function and estimating functions are directly tied to achieving covariate balance between weighted treatment groups. We develop practical numerical algorithms for computing the regularized calibrated estimators with group Lasso by innovatively exploiting Fisher scoring, and provide rigorous high-dimensional analysis for the resulting augmented IPW estimators under suitable sparsity conditions, while tackling technical issues absent or overlooked in previous analyses. We present simulation studies and an empirical application to estimate the effects of maternal smoking on birth weights. The proposed methods are implemented in the R package mRCAL.

stat.ME↗

arXiv2021

On Irreversible Metropolis Sampling Related to Langevin Dynamics

There has been considerable interest in designing Markov chain Monte Carlo algorithms by exploiting numerical methods for Langevin dynamics, which includes Hamiltonian dynamics as a deterministic case. A prominent approach is Hamiltonian Monte Carlo (HMC), where a leapfrog discretization of Hamiltonian dynamics is employed. We investigate a recently proposed class of irreversible sampling algorithms, called Hamiltonian assisted Metropolis sampling (HAMS), which uses an augmented target density similarly as in HMC, but involves a flexible proposal scheme and a carefully formulated acceptance-rejection scheme to achieve generalized reversibility. We show that as the step size tends to 0, the HAMS proposal satisfies a class of stochastic differential equations including Langevin dynamics as a special case. We provide theoretical results for HAMS under the univariate Gaussian setting, including the stationary variance, the expected acceptance rate, and the spectral radius. From these results, we derive default choices of tuning parameters for HAMS, such that only the step size needs to be tuned in applications. Various relatively recent algorithms for Langevin dynamics are also shown to fall in the class of HAMS proposals up to negligible differences. Our numerical experiments on sampling high-dimensional latent variables confirm that the HAMS algorithms consistently achieve superior performance, compared with several Metropolis-adjusted algorithms based on popular integrators of Langevin dynamics.

stat.CO↗

arXiv2021

Model-Assisted Inference for Covariate-Specific Treatment Effects with High-dimensional Data

Covariate-specific treatment effects (CSTEs) represent heterogeneous treatment effects across subpopulations defined by certain selected covariates. In this article, we consider marginal structural models where CSTEs are linearly represented using a set of basis functions of the selected covariates. We develop a new approach in high-dimensional settings to obtain not only doubly robust point estimators of CSTEs, but also model-assisted confidence intervals, which are valid when a propensity score model is correctly specified but an outcome regression model may be misspecified. With a linear outcome model and subpopulations defined by discrete covariates, both point estimators and confidence intervals are doubly robust for CSTEs. In contrast, confidence intervals from existing high-dimensional methods are valid only when both the propensity score and outcome models are correctly specified. We establish asymptotic properties of the proposed point estimators and the associated confidence intervals. We present simulation studies and empirical applications which demonstrate the advantages of the proposed method compared with competing ones.

stat.ME↗

arXiv2021

On loss functions and regret bounds for multi-category classification

We develop new approaches in multi-class settings for constructing proper scoring rules and hinge-like losses and establishing corresponding regret bounds with respect to the zero-one or cost-weighted classification loss. Our construction of losses involves deriving new inverse mappings from a concave generalized entropy to a loss through the use of a convex dissimilarity function related to the multi-distribution $f$-divergence. Moreover, we identify new classes of multi-class proper scoring rules, which also recover and reveal interesting relationships between various composite losses currently in use. We establish new classification regret bounds in general for multi-class proper scoring rules by exploiting the Bregman divergences of the associated generalized entropies, and, as applications, provide simple meaningful regret bounds for two specific classes of proper scoring rules. Finally, we derive new hinge-like convex losses, which are tighter convex extensions than related hinge-like losses and geometrically simpler with fewer non-differentiable edges, while achieving similar regret bounds. We also establish a general classification regret bound for all losses which induce the same generalized entropy as the zero-one loss.

math.ST↗

arXiv2020

Consistent and robust inference in hazard probability and odds models with discrete-time survival data

For discrete-time survival data, conditional likelihood inference in Cox's hazard odds model is theoretically desirable but exact calculation is numerical intractable with a moderate to large number of tied events. Unconditional maximum likelihood estimation over both regression coefficients and baseline hazard probabilities can be problematic with a large number of time intervals. We develop new methods and theory using numerically simple estimating functions, along with model-based and model-robust variance estimation, in hazard probability and odds models. For the probability hazard model, we derive as a consistent estimator the Breslow-Peto estimator, previously known as an approximation to the conditional likelihood estimator in the hazard odds model. For the odds hazard model, we propose a weighted Mantel-Haenszel estimator, which satisfies conditional unbiasedness given the numbers of events in addition to the risk sets and covariates, similarly to the conditional likelihood estimator. Our methods are expected to perform satisfactorily in a broad range of settings, with small or large numbers of tied events corresponding to a large or small number of time intervals. The methods are implemented in the R package dSurvival.

stat.ME↗

arXiv2020

Doubly Robust Semiparametric Inference Using Regularized Calibrated Estimation with High-dimensional Data

Consider semiparametric estimation where a doubly robust estimating function for a low-dimensional parameter is available, depending on two working models. With high-dimensional data, we develop regularized calibrated estimation as a general method for estimating the parameters in the two working models, such that valid Wald confidence intervals can be obtained for the parameter of interest under suitable sparsity conditions if either of the two working models is correctly specified. We propose a computationally tractable two-step algorithm and provide rigorous theoretical analysis which justifies sufficiently fast rates of convergence for the regularized calibrated estimators in spite of sequential construction and establishes a desired asymptotic expansion for the doubly robust estimator. As concrete examples, we discuss applications to partially linear, log-linear, and logistic models and estimation of average treatment effects. Numerical studies in the former three examples demonstrate superior performance of our method, compared with debiased Lasso.

stat.ME↗

arXiv2020

High-dimensional Model-assisted Inference for Local Average Treatment Effects with Instrumental Variables

Consider the problem of estimating the local average treatment effect with an instrument variable, where the instrument unconfoundedness holds after adjusting for a set of measured covariates. Several unknown functions of the covariates need to be estimated through regression models, such as instrument propensity score and treatment and outcome regression models. We develop a computationally tractable method in high-dimensional settings where the numbers of regression terms are close to or larger than the sample size. Our method exploits regularized calibrated estimation, which involves Lasso penalties but carefully chosen loss functions for estimating coefficient vectors in these regression models, and then employs a doubly robust estimator for the treatment parameter through augmented inverse probability weighting. We provide rigorous theoretical analysis to show that the resulting Wald confidence intervals are valid for the treatment parameter under suitable sparsity conditions if the instrument propensity score model is correctly specified, but the treatment and outcome regression models may be misspecified. For existing high-dimensional methods, valid confidence intervals are obtained for the treatment parameter if all three models are correctly specified. We evaluate the proposed methods via extensive simulation studies and an empirical application to estimate the returns to education.

stat.ME↗

arXiv2020

Hamiltonian Assisted Metropolis Sampling

Various Markov chain Monte Carlo (MCMC) methods are studied to improve upon random walk Metropolis sampling, for simulation from complex distributions. Examples include Metropolis-adjusted Langevin algorithms, Hamiltonian Monte Carlo, and other recent algorithms related to underdamped Langevin dynamics. We propose a broad class of irreversible sampling algorithms, called Hamiltonian assisted Metropolis sampling (HAMS), and develop two specific algorithms with appropriate tuning and preconditioning strategies. Our HAMS algorithms are designed to achieve two distinctive properties, while using an augmented target density with momentum as an auxiliary variable. One is generalized detailed balance, which induces an irreversible exploration of the target. The other is a rejection-free property, which allows our algorithms to perform satisfactorily with relatively large step sizes. Furthermore, we formulate a framework of generalized Metropolis--Hastings sampling, which not only highlights our construction of HAMS at a more abstract level, but also facilitates possible further development of irreversible MCMC algorithms. We present several numerical experiments, where the proposed algorithms are found to consistently yield superior results among existing ones.

stat.CO↗

arXiv2019

Analysis of odds, probability, and hazard ratios: From 2 by 2 tables to two-sample survival data

Analysis of 2 by 2 tables and two-sample survival data has been widely used. Exact calculation is computational intractable for conditional likelihood inference in odds ratio models with large marginals in 2 by 2 tables, or partial likelihood inference in Cox's proportional hazards models with considerable tied event times. Approximate methods are often employed, but their statistical properties have not been formally studied while taking into account the approximation involved. We develop new methods and theory by constructing suitable estimating functions while leveraging knowledge from conditional or partial likelihood inference. We propose a weighted Mantel--Haenszel estimator in an odds ratio model such as Cox's discrete-time proportional hazards model. Moreover, we consider a probability ratio model, and derive as a consistent estimator the Breslow--Peto estimator, which has been regarded as an approximation to partial likelihood estimation in the odds ratio model. We study both model-based and model-robust variance estimation. For the Breslow--Peto estimator, our new model-based variance estimator is no greater than the commonly reported variance estimator. We present numerical studies which support the theoretical findings.

stat.ME↗

arXiv2019

Semi-supervised Logistic Learning Based on Exponential Tilt Mixture Models

Consider semi-supervised learning for classification, where both labeled and unlabeled data are available for training. The goal is to exploit both datasets to achieve higher prediction accuracy than just using labeled data alone. We develop a semi-supervised logistic learning method based on exponential tilt mixture models, by extending a statistical equivalence between logistic regression and exponential tilt modeling. We study maximum nonparametric likelihood estimation and derive novel objective functions which are shown to be Fisher consistent. We also propose regularized estimation and construct simple and highly interpretable EM algorithms. Finally, we present numerical results which demonstrate the advantage of the proposed methods compared with existing methods.

stat.ML↗

arXiv2019

Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression

For multivariate nonparametric regression, functional analysis-of-variance (ANOVA) modeling aims to capture the relationship between a response and covariates by decomposing the unknown function into various components, representing main effects, two-way interactions, etc. Such an approach has been pursued explicitly in smoothing spline ANOVA modeling and implicitly in various greedy methods such as MARS. We develop a new method for functional ANOVA modeling, based on doubly penalized estimation using total-variation and empirical-norm penalties, to achieve sparse selection of component functions and their knots. For this purpose, we formulate a new class of hierarchical total variations, which measures total variations at different levels including main effects and multi-way interactions, possibly after some order of differentiation. Furthermore, we derive suitable basis functions for multivariate splines such that the hierarchical total variation can be represented as a regular Lasso penalty, and hence we extend a previous backfitting algorithm to handle doubly penalized estimation for ANOVA modeling. We present extensive numerical experiments on simulations and real data to compare our method with existing methods including MARS, tree boosting, and random forest. The results are very encouraging and demonstrate considerable gains from our method in both prediction or classification accuracy and simplicity of the fitted functions.

stat.ME↗

arXiv2019

On doubly robust estimation for logistic partially linear models

Consider a logistic partially linear model, in which the logit of the mean of a binary response is related to a linear function of some covariates and a nonparametric function of other covariates. We derive simple, doubly robust estimators of coefficient for the covariates in the linear component of the partially linear model. Such estimators remain consistent if either a nuisance model is correctly specified for the nonparametric component, or another nuisance model is correctly specified for the means of the covariates of interest given other covariates and the response at a fixed value. In previous works, conditional density models are needed for the latter purposes unless a scalar, binary covariate is handled. We also propose two specific doubly robust estimators: one is locally-efficient like in our class of doubly robust estimators and the other is numerically and statistically simpler and can achieve reasonable efficiency especially when the true coefficients are close to 0.

stat.ME↗

arXiv2018

Improved Methods for Moment Restriction Models with Marginally Incompatible Data Combination and an Application to Two-sample Instrumental Variable Estimation

Combining information from multiple samples is often needed in biomedical and economic studies, but the differences between these samples must be appropriately taken into account in the analysis of the combined data. We study estimation for moment restriction models with data combination from two samples under an ignorablility-type assumption but allowing for different marginal distributions of common variables between the two samples. Suppose that an outcome regression model and a propensity score model are specified. By leveraging the semiparametric efficiency theory, we derive an augmented inverse probability weighted (AIPW) estimator that is locally efficient and doubly robust with respect to the outcome regression and propensity score models. Furthermore, we develop calibrated regression and likelihood estimators that are not only locally efficient and doubly robust, but also intrinsically efficient in achieving smaller variances than the AIPW estimator when the propensity score model is correctly specified but the outcome regression model may be misspecified. As an important application, we study the two-sample instrumental variable problem and derive the corresponding estimators while allowing for incompatible distributions of common variables between the two samples. Finally, we provide a simulation study and an econometric application on public housing projects to demonstrate the superior performance of our improved estimators.

stat.ME↗

arXiv2018

Improved Estimation of Average Treatment Effects on the Treated: Local Efficiency, Double Robustness, and Beyond

Estimation of average treatment effects on the treated (ATT) is an important topic of causal inference in econometrics and statistics. This problem seems to be often treated as a simple modification or extension of that of estimating overall average treatment effects (ATE). However, the propensity score is no longer ancillary for estimation of ATT, in contrast with estimation of ATE. In this article, we review semiparametric theory for estimation of ATT and the use of efficient influence functionsto derive augmented inverse probability weighted (AIPW) estimators that are locally efficient and doubly robust. Moreover, we discuss improved estimation over AIPW by developing calibrated regression and likelihood estimators that are not only locally efficient and doubly robust, but also intrinsically efficient in achieving smaller variances than AIPW estimators when a propensity score model is correctly specified but an outcome regression model may be misspecified. Finally, we present two simulation studies and an econometric application to demonstrate the advantage of the proposed methods when compared with existing methods.

stat.ME↗

arXiv2018

Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data

Consider the problem of estimating average treatment effects when a large number of covariates are used to adjust for possible confounding through outcome regression and propensity score models. The conventional approach of model building and fitting iteratively can be difficult to implement, depending on ad hoc choices of what variables are included. In addition, uncertainty from the iterative process of model selection is complicated and often ignored in subsequent inference about treatment effects. We develop new methods and theory to obtain not only doubly robust point estimators for average treatment effects, which remain consistent if either the propensity score model or the outcome regression model is correctly specified, but also model-assisted confidence intervals, which are valid when the propensity score model is correctly specified but the outcome regression model may be misspecified. With a linear outcome model, the confidence intervals are doubly robust, that is, being also valid when the outcome model is correctly specified but the propensity score model may be misspecified. Our methods involve regularized calibrated estimators with Lasso penalties, but carefully chosen loss functions, for fitting propensity score and outcome regression models. We provide high-dimensional analysis to establish the desired properties of our methods under comparable conditions to previous results, which give valid confidence intervals when both the propensity score and outcome regression are correctly specified. We present a simulation study and an empirical application which confirm the advantages of the proposed methods compared with related methods based on regularized maximum likelihood estimation.

math.ST↗