arXiv subjects

Jerome P. Reiter

Publications and source records attributed to Jerome P. Reiter.

At least 19 recordsLinked to original sources

arXiv2026

Differentially Private Tests of Fisher's Sharp Null Hypothesis for Binary Outcomes

Across many disciplines, causal inference often relies on randomized experiments with binary outcomes. In such experiments, often analysts are interested in testing Fisher's sharp null hypothesis, i.e., the treatment has zero effect on the outcome for every study subject. Sometimes the outcomes are sensitive and must be kept confidential, for example, when they comprise physical or mental health measurements. Releasing test statistics or p-values computed with the confidential outcomes can leak information about the individuals in the study. Those responsible for sharing the analysis results may wish to bound this information leakage, which they can do by ensuring the released outputs satisfy differential privacy. In this article, we develop several differentially private tests of Fisher's sharp null for binary outcomes. Specifically, we consider direct perturbation approaches that inject calibrated noise into test statistics or p-values, as well as a Bayesian denoising framework that explicitly models the privacy mechanism. We further develop decision-making procedures under privacy constraints, including a Bayes risk-optimal rule and a frequentist-calibrated significance test. Through theoretical results, simulation studies, and an application to the ADAPTABLE clinical trial, we demonstrate that our methods can achieve valid and interpretable causal inference while ensuring the differential privacy guarantee.

stat.ME↗

arXiv2026

Differentially Private Bayesian Inference for Gaussian Copula Correlations

Gaussian copulas are widely used to estimate multivariate distributions and relationships. We present algorithms for estimating Gaussian copula correlations that ensure differential privacy. We first convert data values into sets of two-way tables of counts above and below marginal medians. We then add noise to these counts to satisfy differential privacy. We utilize the one-to-one correspondence between the true counts and the copula correlation to estimate a posterior distribution of the copula correlation given the noisy counts, marginalizing over the distribution of the underlying true counts using a composite likelihood. We also present an alternative, maximum likelihood approach for point estimation. Using simulation studies, we compare these methods to extant methods in the literature for computing differentially private copula correlations.

stat.ME↗

arXiv2026

Multiple imputation for nonresponse in surveys using design weights and auxiliary margins

Survey data typically have missing values due to unit and item nonresponse. Sometimes, survey organizations know the marginal distributions of certain categorical variables in the target population. As shown in previous work, survey organizations can leverage these distributions in multiple imputation for nonignorable unit non-response, generating imputations that result in plausible completed-data estimates for the variables with known margins. However, this prior work does not use the design weights for unit nonrespondents. We extend this previous work to utilize the design weights for all sampled units. We illustrate the approach using simulation studies.

stat.ME↗

arXiv2026

Setting the Privacy Budget in Differential Privacy by Bounding Adversaries' Odds of Learning Sensitive Information

Differential privacy is a mathematical definition of what it means to protect data subjects' privacy in data releases. Differential privacy depends on a parameter $ε$ known as the privacy budget. The value of $\varepsilon$ determines the nature of the privacy guarantee, with smaller values generally offering more privacy. However, reducing $\varepsilon$ also tends to decrease the accuracy of results protected with differentially private algorithms. Setting a value for $\varepsilon$ that satisfactorily balances this risk/accuracy trade off is complicated in practice, and there is not a standard approach to doing so. In part this is because practitioners may struggle to understand the privacy guarantee afforded by $\varepsilon$. We present an approach to interpreting and setting $\varepsilon$ in which (i) the practitioner establishes bounds on the posterior odds that adversaries can learn sensitive information, and (ii) the practitioner converts these bounds to values of $\varepsilon$. We illustrate the approach using data from a case control study.

stat.ME↗

arXiv2026

Interval Estimation for Binomial Proportions Under Differential Privacy

When releasing binary proportions computed using sensitive data, several government agencies and other data stewards protect confidentiality of the underlying values by ensuring the released statistics satisfy differential privacy. Typically, this is done by adding carefully chosen noise to the sample proportion computed using the confidential data. In this article, we describe and compare methods for turning this differentially private proportion into an interval estimate for an underlying population probability. Specifically, we consider differentially private versions of the Wald and Wilson intervals, Bayesian credible intervals based on denoising the differentially private proportion, and an exact interval motivated by the Clopper-Pearson confidence interval. We examine the repeated sampling performances of the intervals using simulation studies under both the Laplace mechanism and the discrete Gaussian mechanism across a range of privacy guarantees. We find that while several methods can offer reasonable performances, the Bayesian credible intervals are the most attractive.

stat.ME↗

arXiv2026

Multiple Imputation Diagnostics when using Electronic Health Record Data in Observational Studies: A Case Study

Missing values in electronic health record (EHR) data pose a significant challenge for epidemiologic research. Traditional methods for handling missing data, like mean imputation, may introduce bias. Multiple imputation (MI) offers a principled solution by generating multiple plausible values based on statistical models. However, MI requires careful model specification and validation of imputations, ideally using multivariate graphical tools. We demonstrate the application of such tools to validate MI in a study of chronic kidney disease, assessing cardiovascular outcomes linked to neighborhood socioeconomic status (nSES). This study used data from Duke University Health System (DUHS) and Lincoln Community Health Center (LCHC). Eligible patients had at least one encounter within DUHS or LCHC and had two estimated glomerular filtration rate (eGFR) values <60 mL/min per 1.73 m2 more than 90 days apart between January 1, 2007 and July 1, 2008. Socioeconomic status was assessed using the Agency for Healthcare Research and Quality (AHRQ) index based on census data. The main outcome was a cardiovascular disease-related hospitalization. Participants were mostly older (mean age 73 years), female (64%), and Black (43%). Participants living in lower nSES neighborhoods had higher mean systolic blood pressure (SBP: 140 mmHg) and hemoglobin A1c (HbA1c) levels (7.1%) as compared to participants living in higher nSES neighborhoods. A machine learning based approach, Classification and Regression Trees (CART), was the preferred approach to impute missing data. The distributions of imputed values of SBP and HbA1c were impacted by whether marginal or conditional values of SBP and HbA1c were imputed. The choice of MI had minimal impact on inference and prediction. Future research may want to extend our results and consider how results may differ when using EHR data from multiple health systems.

stat.ME↗

arXiv2025

Differentially Private Computation of the Gini Index for Income Inequality

The Gini index is a widely reported measure of income inequality. In some settings, the underlying data used to compute the Gini index are confidential. The organization charged with reporting the Gini index may be concerned that its release could leak information about the underlying data. We present an approach for bounding this information leakage by releasing a differentially private version of the Gini index. In doing so, we analyze how adding, deleting, or altering a single observation in any specific dataset can affect the computation of the Gini index; this is known as the local sensitivity. We then derive a smooth upper bound on the local sensitivity. Using this bound, we define a mechanism that adds noise to the Gini index, thereby satisfying differential privacy. Using simulated and genuine income data, we show that the mechanism can reduce the errors from noise injection substantially relative to differentially private algorithms that rely on the global sensitivity, that is, the maximum of the local sensitivities over all possible datasets. We characterize settings where using smooth sensitivity can provide highly accurate estimates, as well as settings where the noise variance is simply too large to provide reliably useful results. We also present a Bayesian post-processing step that provides interval estimates about the value of the Gini index computed with the confidential data.

stat.ME↗

arXiv2025

Regression Analysis After Bipartite Bayesian Record Linkage

In many settings, a data curator links records from two files to produce datasets that are shared with secondary analysts. Analysts use the linked files to estimate models of interest, such as regressions. Such two-stage approaches do not necessarily account for uncertainty in model parameters that results from uncertainty in the linkages. Further, they do not leverage the relationships among the study variables in the two files to help determine the linkages. We propose a multiple imputation framework to address these shortcomings. First, we use a bipartite Bayesian record linkage model to generate multiple plausible linked datasets, disregarding the information in the study variables. Second, we presume each linked file has a mixture of true links and false links. We estimate the mixture model using information from the study variables. Through simulation studies under a regression setting, we demonstrate that estimates of the regression model parameters can be more accurate than those based on an analogous two-stage approach. We illustrate the integrated approach using data from the Survey on Household Income and Wealth, examining a regression involving the persistence of income.

stat.ME↗

arXiv2025

A Sensitivity Analysis Framework for Quantifying Confidence in Decisions in the Presence of Data Uncertainty

Nearly all statistical analyses that inform policy-making are based on imperfect data. As examples, the data may suffer from measurement errors, missing values, sample selection bias, or record linkage errors. Analysts have to decide how to handle such data imperfections, e.g., analyze only the complete cases or impute values for the missing items via some posited model. Their choices can influence estimates and hence, ultimately, policy decisions. Thus, it is prudent for analysts to evaluate the sensitivity of estimates and policy decisions to the assumptions underlying their choices. To facilitate this goal, we propose that analysts define metrics and visualizations that target the sensitivity of the ultimate decision to the assumptions underlying their approach to handling the data imperfections. Using these visualizations, the analyst can assess their confidence in the policy decision under their chosen analysis. We illustrate metrics and corresponding visualizations with two examples, namely considering possible measurement error in the inputs of predictive models of presidential vote share and imputing missing values when evaluating the percentage of children exposed to high levels of lead.

stat.ME↗

arXiv2025

Outcome-Assisted Multiple Imputation of Missing Treatments

We provide guidance on multiple imputation of missing at random treatments in observational studies. Specifically, analysts should account for both covariates and outcomes, i.e., not just use propensity scores, when imputing the missing treatments. To do so, we develop outcome-assisted multiple imputation of missing treatments: the analyst fits a regression for the outcome on the treatment indicator and covariates, which is used to sharpen the predictive probabilities for missing treatments under an estimated propensity score model. We derive an expression for the bias of the inverse probability weighted estimator for the average treatment effect under multiple imputation of missing treatments, and we show theoretically that this bias can be made small by using outcome-assisted multiple imputation. Simulations demonstrate empirically that outcome-assisted multiple imputation can offer better inferential properties than using the treatment assignment model alone. We illustrate the procedure in an analysis of data from the National Longitudinal Survey of Youth.

stat.ME↗

arXiv2024

Gaussian Copula Models for Nonignorable Missing Data Using Auxiliary Marginal Quantiles

We present an approach for modeling and imputation of nonignorable missing data. Our approach uses Bayesian data integration to combine (1) a Gaussian copula model for all study variables and missingness indicators, which allows arbitrary marginal distributions, nonignorable missingess, and other dependencies, and (2) auxiliary information in the form of marginal quantiles for some study variables. We prove that, remarkably, one only needs a small set of accurately-specified quantiles to estimate the copula correlation consistently. The remaining marginal distribution functions are inferred nonparametrically and jointly with the copula parameters using an efficient MCMC algorithm. We also characterize the (additive) nonignorable missingness mechanism implied by the copula model. Simulations confirm the effectiveness of this approach for multivariate imputation with nonignorable missing data. We apply the model to analyze associations between lead exposure and end-of-grade test scores for 170,000 North Carolina students. Lead exposure has nonignorable missingness: children with higher exposure are more likely to be measured. We elicit marginal quantiles for lead exposure using statistics provided by the Centers for Disease Control and Prevention. Multiple imputation inferences under our model support stronger, more adverse associations between lead exposure and educational outcomes relative to complete case and missing-at-random analyses.

stat.ME↗

arXiv2024

Bayesian Inference Under Differential Privacy With Bounded Data

We describe Bayesian inference for the parameters of Gaussian models of bounded data protected by differential privacy. Using this setting, we demonstrate that analysts can and should take constraints imposed by the bounds into account when specifying prior distributions. Additionally, we provide theoretical and empirical results regarding what classes of default priors produce valid inference for a differentially private release in settings where substantial prior information is not available. We discuss how these results can be applied to Bayesian inference for regression with differentially private data.

stat.ME↗

arXiv2024

Differentially Private Estimation of Weighted Average Treatment Effects for Binary Outcomes

In the social and health sciences, researchers often make causal inferences using sensitive variables. These researchers, as well as the data holders themselves, may be ethically and perhaps legally obligated to protect the confidentiality of study participants' data. It is now known that releasing any statistics, including estimates of causal effects, computed with confidential data leaks information about the underlying data values. Thus, analysts may desire to use causal estimators that can provably bound this information leakage. Motivated by this goal, we develop algorithms for estimating weighted average treatment effects with binary outcomes that satisfy the criterion of differential privacy. We present theoretical results on the accuracy of several differentially private estimators of weighted average treatment effects. We illustrate the empirical performance of these estimators using simulated data and a causal analysis using data on education and income.

stat.ME↗

arXiv2024

Evaluating Binary Outcome Classifiers Estimated from Survey Data

Surveys are commonly used to facilitate research in epidemiology, health, and the social and behavioral sciences. Often, these surveys are not simple random samples, and respondents are given weights reflecting their probability of selection into the survey. It is well known that analysts can use these survey weights to produce unbiased estimates of population quantities like totals. In this article, we show that survey weights also can be beneficial for evaluating the quality of predictive models when splitting data into training and test sets. In particular, we characterize model assessment statistics, such as sensitivity and specificity, as finite population quantities, and compute survey-weighted estimates of these quantities with sample test data comprising a random subset of the original data.Using simulations with data from the National Survey on Drug Use and Health and the National Comorbidity Survey, we show that unweighted metrics estimated with sample test data can misrepresent population performance, but weighted metrics appropriately adjust for the complex sampling design. We also show that this conclusion holds for models trained using upsampling for mitigating class imbalance. The results suggest that weighted metrics should be used when evaluating performance on sample test data.

stat.ME↗

arXiv2024

Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Framework

Commonly, AI or machine learning (ML) models are evaluated on benchmark datasets. This practice supports innovative methodological research, but benchmark performance can be poorly correlated with performance in real-world applications -- a construct validity issue. To improve the validity and practical usefulness of evaluations, we propose using an estimands framework adapted from international clinical trials guidelines. This framework provides a systematic structure for inference and reporting in evaluations, emphasizing the importance of a well-defined estimation target. We illustrate our proposal on examples of commonly used evaluation methodologies - involving cross-validation, clustering evaluation, and LLM benchmarking - that can lead to incorrect rankings of competing models (rank reversals) with high probability, even when performance differences are large. We demonstrate how the estimands framework can help uncover underlying issues, their causes, and potential solutions. Ultimately, we believe this framework can improve the validity of evaluations through better-aligned inference, and help decision-makers and model users interpret reported results more effectively.

cs.LG↗

arXiv2024

Imputation of Nonignorable Missing Data in Surveys Using Auxiliary Margins Via Hot Deck and Sequential Imputation

Survey data collection often is plagued by unit and item nonresponse. To reduce reliance on strong assumptions about the missingness mechanisms, statisticians can use information about population marginal distributions known, for example, from censuses or administrative databases. One approach that does so is the Missing Data with Auxiliary Margins, or MD-AM, framework, which uses multiple imputation for both unit and item nonresponse so that survey-weighted estimates accord with the known marginal distributions. However, this framework relies on specifying and estimating a joint distribution for the survey data and nonresponse indicators, which can be computationally and practically daunting in data with many variables of mixed types. We propose two adaptations to the MD-AM framework to simplify the imputation task. First, rather than specifying a joint model for unit respondents' data, we use random hot deck imputation while still leveraging the known marginal distributions. Second, instead of sampling from conditional distributions implied by the joint model for the missing data due to item nonresponse, we apply multiple imputation by chained equations for item nonresponse before imputation for unit nonresponse. Using simulation studies with nonignorable missingness mechanisms, we demonstrate that the proposed approach can provide more accurate point and interval estimates than models that do not leverage the auxiliary information. We illustrate the approach using data on voter turnout from the U.S. Current Population Survey.

stat.ME↗

arXiv2024

Prior-itizing Privacy: A Bayesian Approach to Setting the Privacy Budget in Differential Privacy

When releasing outputs from confidential data, agencies need to balance the analytical usefulness of the released data with the obligation to protect data subjects' confidentiality. For releases satisfying differential privacy, this balance is reflected by the privacy budget, $\varepsilon$. We provide a framework for setting $\varepsilon$ based on its relationship with Bayesian posterior probabilities of disclosure. The agency responsible for the data release decides how much posterior risk it is willing to accept at various levels of prior risk, which implies a unique $\varepsilon$. Agencies can evaluate different risk profiles to determine one that leads to an acceptable trade-off in risk and utility.

cs.CR↗

arXiv2024

Fully Synthetic Data for Complex Surveys

When seeking to release public use files for confidential data, statistical agencies can generate fully synthetic data. We propose an approach for making fully synthetic data from surveys collected with complex sampling designs. Our approach adheres to the general strategy proposed by Rubin (1993). Specifically, we generate pseudo-populations by applying the weighted finite population Bayesian bootstrap to account for survey weights, take simple random samples from those pseudo-populations, estimate synthesis models using these simple random samples, and release simulated data drawn from the models as public use files. To facilitate variance estimation, we use the framework of multiple imputation with two data generation strategies. In the first, we generate multiple data sets from each simple random sample. In the second, we generate a single synthetic data set from each simple random sample. We present multiple imputation combining rules for each setting. We illustrate the repeated sampling properties of the combining rules via simulation studies, including comparisons with synthetic data generation based on pseudo-likelihood methods. We apply the proposed methods to a subset of data from the American Community Survey.

stat.ME↗