arXiv subjects

Matthew R. Williams

Publications and source records attributed to Matthew R. Williams.

At least 19 recordsLinked to original sources

arXiv2026

Design-Based Supervised Learning with Noisy Human Labels

Researchers increasingly use automated classifiers to label unstructured data for statistical analysis. Existing rectification methods can correct errors in these automated labels using a probability-sampled audit set, but they usually treat the audit labels as correct. In practice, human audit labels are often noisy, and only some audited items are reviewed by an expert or adjudicator. We propose Partially Adjudicated Design-Based Supervised Learning (PA-DSL), a method for this setting. It uses adjudicated cases to correct noisy human labels and then uses the corrected audit information to debias analyses based on the full set of automated labels. The estimator is valid for a broad class of downstream analyses when the audit and adjudication probabilities are known. In synthetic and Wikipedia Detox semi-synthetic experiments, PA-DSL maintains nominal coverage and reduces RMSE by 10-17% relative to using only adjudicated labels when noisy human labels contain recoverable signal.

stat.ML↗

arXiv2026

Design Effect Ratios for Bayesian Survey Models: A Diagnostic Framework for Identifying Survey-Sensitive Parameters

Bayesian hierarchical models are increasingly fitted to complex survey data through weighted pseudo-posteriors, with a post-processing step that rescales the posterior to match a design-based sandwich covariance. Applied to all parameters at once, this correction can be harmful as well as protective: parameters stabilized by hierarchical shrinkage or identified by between-group variation have their credible intervals needlessly widened or spuriously narrowed. We propose the Design Effect Ratio (DER), the ratio of a parameter's design-based sandwich variance, computed for a declared variance target, to its model-based posterior variance, as a per-parameter diagnostic for applying the correction selectively. For hierarchical Gaussian models we derive exact finite-sample expressions under explicit balance and common-design-effect hypotheses: fixed-effect DERs scale with the design effect attenuated by the between-group share of identifying variation, and random-effect DERs factor into design effect, shrinkage, and a group-count term, with a conservation identity linking the two levels. A general matrix formula covers arbitrary parameter blocks and non-Gaussian likelihoods. A simulation study with a genuine informative two-stage sampling mechanism and a re-analysis of the 2019 National Survey of Early Care and Education, reported under both the design-PSU and model-group variance targets, show the diagnostic separating survey-sensitive from shrinkage-protected parameters and the selective correction avoiding the damage of blanket rescaling. The R package svyder implements the workflow.

stat.ME↗

arXiv2026

Privacy Amplification for Synthetic data using Range Restriction

We introduce a new class of range restricted formal data privacy standards that condition on owner beliefs about sensitive data ranges. By incorporating this additional information, we can provide a stronger privacy guarantee (e.g. an amplification). The range restricted formal privacy standards protect only a subset (or ball) of data values and exclude ranges (or balls) believed to be already publicly known. The privacy standards are designed for the risk-weighted pseudo posterior (model) mechanism (PPM) used to generate synthetic data under an asymptotic Differential (aDP) privacy guarantee. The PPM downweights the likelihood contribution for each record proportionally to its disclosure risk. The PPM is adapted under inclusion of beliefs by adjusting the risk-weighted pseudo likelihood. We introduce two alternative adjustments. The first expresses data owner knowledge of the sensitive range as a probability, $\lambda$, that a datum value drawn from the underlying generating distribution lies outside the ball or subspace of values that are sensitive. The portion of each datum likelihood contribution deemed sensitive is then $(1-\lambda) \leq 1$ and is the only portion of the likelihood subject to risk down-weighting. The second adjustment encodes knowledge as the difference in probability masses $P(R) \leq 1$ between the edges of the sensitive range, $R$. We use the resulting conditional (pseudo) likelihood for a sensitive record, which boosts its worst case tail values away from 0. We compare privacy and utility properties for the PPM under the aDP and range restricted privacy standards.

stat.ME↗

arXiv2025

Uncertainty Quantification for Multi-level Models Using the Survey-Weighted Pseudo-Posterior

Parameter estimation and inference from complex survey samples typically focuses on global model parameters whose estimators have asymptotic properties, such as from fixed effects regression models. The central challenge is to both mitigate bias induced from potentially unbalanced samples and to incorporate adjustments for differences in effective sample size to get correct variance and interval estimates. We present a motivating example of Bayesian inference for a multi-level or mixed effects model in which estimates of both the local parameters (e.g. group level random effects) and the global parameters need to be adjusted for the complex sampling design. We evaluate the limitations of the survey-weighted pseudo-posterior and an existing automated post-processing method to improve the uncertainty quantification. We propose modifications to the automated process and demonstrate their improvements for multi-level models via a simulation study and a motivating example from the National Survey on Drug Use and Health. Reproduction examples are available from the authors and the updated R package is available via github:https://github.com/RyanHornby/csSampling

stat.ME↗

arXiv2025

Bayesian Pseudo Posterior Mechanism for Differentially Private Machine Learning

Differential privacy (DP) is becoming increasingly important for deployed machine learning applications because it provides strong guarantees for protecting the privacy of individuals whose data is used to train models. However, DP mechanisms commonly used in machine learning tend to struggle on many real world distributions, including highly imbalanced or small labeled training sets. In this work, we propose a new scalable DP mechanism for deep learning models, SWAG-PPM, by using a pseudo posterior distribution that downweights by-record likelihood contributions proportionally to their disclosure risks as the randomized mechanism. As a motivating example from official statistics, we demonstrate SWAG-PPM on a workplace injury text classification task using a highly imbalanced public dataset published by the U.S. Occupational Safety and Health Administration (OSHA). We find that SWAG-PPM exhibits only modest utility degradation against a non-private comparator while greatly outperforming the industry standard DP-SGD for a similar privacy budget.

stat.ML↗

arXiv2025

Representative dietary behavior patterns and associations with cardiometabolic outcomes in Puerto Rico using a Bayesian latent class analysis for non-probability samples

There is limited understanding of how dietary behaviors cluster together and influence cardiometabolic health at a population level in Puerto Rico. Data availability is scarce, particularly outside of urban areas, and is often limited to non-probability sample (NPS) data where sample inclusion mechanisms are unknown. In order to generalize results to the broader Puerto Rican population, adjustments are necessary to account for selection bias but are difficult to implement for NPS data. Although Bayesian latent class models enable summaries of dietary behavior variables through underlying patterns, they have not yet been adapted to the NPS setting. We propose a novel Weighted Overfitted Latent Class Analysis for Non-probability samples (WOLCAN). WOLCAN utilizes a quasi-randomization framework to (1) model pseudo-weights for an NPS using Bayesian additive regression trees (BART) and a reference probability sample, and (2) integrate the pseudo-weights within a weighted pseudo-likelihood approach for Bayesian latent class analysis, while propagating pseudo-weight uncertainty into parameter estimation. A stacked sample approach is used to allow shared individuals between the NPS and the reference sample. We evaluate model performance through simulations and apply WOLCAN to data from the Puerto Rico Observational Study of Psychosocial, Environmental, and Chronic Disease Trends (PROSPECT). We identify dietary behavior patterns for adults in Puerto Rico aged 30 to 75 and examine their associations with type 2 diabetes, hypertension, and hypercholesterolemia. Our findings suggest that an out-of-home eating pattern is associated with a higher likelihood of these cardiometabolic outcomes compared to a nutrition-sensitive pattern. WOLCAN effectively reveals generalizable dietary behavior patterns and demonstrates relevant applications in studying diet-disease relationships.

stat.ME↗

arXiv2025

Thresholding Nonprobability Units in Combined Data for Efficient Domain Estimation

Quasi-randomization approaches estimate latent participation probabilities for units from a nonprobability / convenience sample. Estimation of participation probabilities for convenience units allows their combination with units from the randomized survey sample to form a survey weighted domain estimate. One leverages convenience units for domain estimation under the expectation that estimation precision and bias will improve relative to solely using the survey sample; however, convenience sample units that are very different in their covariate support from the survey sample units may inflate estimation bias or variance. This paper develops a method to threshold or exclude convenience units to minimize the variance of the resulting survey weighted domain estimator. We compare our thresholding method with other thresholding constructions in a simulation study for two classes of datasets based on degree of overlap between survey and convenience samples on covariate support. We reveal that excluding convenience units that each express a low probability of appearing in \emph{both} reference and convenience samples reduces estimation error.

stat.ME↗

arXiv2024

Derivation of outcome-dependent dietary patterns for low-income women obtained from survey data using a Supervised Weighted Overfitted Latent Class Analysis

Poor diet quality is a key modifiable risk factor for hypertension and disproportionately impacts low-income women. \sw{Analyzing diet-driven hypertensive outcomes in this demographic is challenging due to the complexity of dietary data and selection bias when the data come from surveys, a main data source for understanding diet-disease relationships in understudied populations. Supervised Bayesian model-based clustering methods summarize dietary data into latent patterns that holistically capture relationships among foods and a known health outcome but do not sufficiently account for complex survey design. This leads to biased estimation and inference and lack of generalizability of the patterns}. To address this, we propose a supervised weighted overfitted latent class analysis (SWOLCA) based on a Bayesian pseudo-likelihood approach that integrates sampling weights into an exposure-outcome model for discrete data. Our model adjusts for stratification, clustering, and informative sampling, and handles modifying effects via interaction terms within a Markov chain Monte Carlo Gibbs sampling algorithm. Simulation studies confirm that the SWOLCA model exhibits good performance in terms of bias, precision, and coverage. Using data from the National Health and Nutrition Examination Survey (2015-2018), we demonstrate the utility of our model by characterizing dietary patterns associated with hypertensive outcomes among low-income women in the United States.

stat.ME↗

arXiv2023

csSampling: An R Package for Bayesian Models for Complex Survey Data

We present csSampling, an R package for estimation of Bayesian models for data collected from complex survey samples. csSampling combines functionality from the probabilistic programming language Stan (via the rstan and brms R packages) and the handling of complex survey data from the survey R package. Under this approach, the user creates a survey-weighted model in brms or provides a custom weighted model via rstan. Survey design information is provided via the svydesign function of the survey package. The cs_sampling function of csSampling estimates the weighted stan model and provides an asymptotic covariance correction for model mis-specification due to using survey sampling weights as plug-in values in the likelihood. This is often known as a ``design effect'' which is the ratio between the variance from a complex survey sample and a simple random sample of the same size. The resulting adjusted posterior draws can then be used for the usual Bayesian inference while also achieving frequentist properties of asymptotic consistency and correct uncertainty (e.g. coverage).

stat.CO↗

arXiv2023

Mechanisms for Global Differential Privacy under Bayesian Data Synthesis

This paper introduces a new method that embeds any Bayesian model used to generate synthetic data and converts it into a differentially private (DP) mechanism. We propose an alteration of the model synthesizer to utilize a censored likelihood that induces upper and lower bounds of [$\exp(-ε/ 2), \exp(ε/ 2)$], where $ε$ denotes the level of the DP guarantee. This censoring mechanism equipped with an $ε-$DP guarantee will induce distortion into the joint parameter posterior distribution by flattening or shifting the distribution towards a weakly informative prior. To minimize the distortion in the posterior distribution induced by likelihood censoring, we embed a vector-weighted pseudo posterior mechanism within the censoring mechanism. The pseudo posterior is formulated by selectively downweighting each likelihood contribution proportionally to its disclosure risk. On its own, the pseudo posterior mechanism produces a weaker asymptotic differential privacy (aDP) guarantee. After embedding in the censoring mechanism, the DP guarantee becomes strict such that it does not rely on asymptotics. We demonstrate that the pseudo posterior mechanism creates synthetic data with the highest utility at the price of a weaker, aDP guarantee, while embedding the pseudo posterior mechanism in the proposed censoring mechanism produces synthetic data with a stronger, non-asymptotic DP guarantee at the cost of slightly reduced utility. The perturbed histogram mechanism is included for comparison.

stat.ME↗

arXiv2023

Methods for Combining Probability and Nonprobability Samples Under Unknown Overlaps

Nonprobability (convenience) samples are increasingly sought to stabilize estimations for one or more population variables of interest that are performed using a randomized survey (reference) sample by increasing the effective sample size. Estimation of a population quantity derived from a convenience sample will typically result in bias since the distribution of variables of interest in the convenience sample is different from the population. A recent set of approaches estimates conditional (on sampling design predictors) inclusion probabilities for convenience sample units by specifying reference sample-weighted pseudo likelihoods. This paper introduces a novel approach that derives the propensity score for the observed sample as a function of conditional inclusion probabilities for the reference and convenience samples as our main result. Our approach allows specification of an exact likelihood for the observed sample. We construct a Bayesian hierarchical formulation that simultaneously estimates sample propensity scores and both conditional and reference sample inclusion probabilities for the convenience sample units. We compare our exact likelihood with the pseudo likelihoods in a Monte Carlo simulation study.

stat.ME↗

arXiv2023

Methods for Combining Probability and Nonprobability Samples Under Unknown Overlaps

Nonprobability (convenience) samples are increasingly sought to reduce the estimation variance for one or more population variables of interest that are estimated using a randomized survey (reference) sample by increasing the effective sample size. Estimation of a population quantity derived from a convenience sample will typically result in bias since the distribution of variables of interest in the convenience sample is different from the population distribution. A recent set of approaches estimates inclusion probabilities for convenience sample units by specifying reference sample-weighted pseudo likelihoods. This paper introduces a novel approach that derives the propensity score for the observed sample as a function of inclusion probabilities for the reference and convenience samples as our main result. Our approach allows specification of a likelihood directly for the observed sample as opposed to the approximate or pseudo likelihood. We construct a Bayesian hierarchical formulation that simultaneously estimates sample propensity scores and the convenience sample inclusion probabilities. We use a Monte Carlo simulation study to compare our likelihood based results with the pseudo likelihood based approaches considered in the literature.

stat.ME↗

arXiv2023

Pseudo Bayesian Estimation of One-way ANOVA Model in Complex Surveys

We devise survey-weighted pseudo posterior distribution estimators under two-stage informative sampling of both primary clusters and secondary nested units for a one-way analysis of variance (ANOVA) population generating model as a simple canonical case where population model random effects are defined to be coincident with the primary clusters, for example student performance based on a survey of schools and students such as the 2000 OECD Programme for International Student Assessment (PISA). We consider estimation on an observed informative sample under both an augmented pseudo likelihood that co-samples the random effects, as well as an integrated likelihood that marginalizes out the random effects from the survey-weighted augmented pseudo likelihood. This paper includes a theoretical exposition that enumerates easily verified conditions for which estimation under the augmented pseudo posterior is guaranteed to be consistent at the true generating parameters. We reveal in simulation that both approaches produce asymptotically unbiased estimation of the generating hyperparameters for the random effects when a key condition on the sum of within cluster weighted residuals is met. We present a comparison with two frequentist alternatives, an expectation-maximization approach and a composite likelihood method that requires pairwise sampling weights.

stat.ME↗

arXiv2022

Re-weighting of Vector-weighted Mechanisms for Utility Maximization under Differential Privacy

We address practical implementation of a risk-weighted pseudo posterior synthesizer for microdata dissemination with a new re-weighting strategy that maximizes utility of released synthetic data under at any level of formal privacy guarantee. Our re-weighting strategy applies to any vector-weighted pseudo posterior mechanism under which a vector of observation-indexed weights are used to downweight likelihood contributions for high disclosure risk records. We demonstrate our method on two different vector-weighted schemes that target high-risk records. Our new method for constructing record-indexed downeighting maximizes the data utility under any privacy budget for the vector-weighted synthesizers by adjusting the by-record weights, such that their individual Lipschitz bounds approach the bound for the entire database. Our method achieves an $(ε= 2 Δ_{\boldsymbolα})-$asymptotic differential privacy (aDP) guarantee, globally, over the space of databases. We illustrate our methods using simulated highly skewed count data and compare the results to a scalar-weighted synthesizer under the Exponential Mechanism (EM). We also apply our methods to a sample of the Survey of Doctorate Recipients and demonstrate the practicality of our methods.

stat.ME↗

arXiv2022

Private Tabular Survey Data Products through Synthetic Microdata Generation

We propose two synthetic microdata approaches to generate private tabular survey data products for public release. We adapt a pseudo posterior mechanism that downweights by-record likelihood contributions with weights $\in [0,1]$ based on their identification disclosure risks to producing tabular products for survey data. Our method applied to an observed survey database achieves an asymptotic global probabilistic differential privacy guarantee. Our two approaches synthesize the observed sample distribution of the outcome and survey weights, jointly, such that both quantities together possess a privacy guarantee. The privacy-protected outcome and survey weights are used to construct tabular cell estimates (where the cell inclusion indicators are treated as known and public) and associated standard errors to correct for survey sampling bias. Through a real data application to the Survey of Doctorate Recipients public use file and simulation studies motivated by the application, we demonstrate that our two microdata synthesis approaches to construct tabular products provide superior utility preservation as compared to the additive-noise approach of the Laplace Mechanism. Moreover, our approaches allow the release of microdata to the public, enabling additional analyses at no extra privacy cost.

stat.ME↗

arXiv2021

Pseudo Bayesian Mixed Models under Informative Sampling

When random effects are correlated with sample design variables, the usual approach of employing individual survey weights (constructed to be inversely proportional to the unit survey inclusion probabilities) to form a pseudo-likelihood no longer produces asymptotically unbiased inference. We construct a weight-exponentiated formulation for the random effects distribution that achieves unbiased inference for generating hyperparameters of the random effects. We contrast our approach with frequentist methods that rely on numerical integration to reveal that only the Bayesian method achieves both unbiased estimation with respect to the sampling design distribution and consistency with respect to the population generating distribution. Our simulations and real data example for a survey of business establishments demonstrate the utility of our approach across different modeling formulations and sampling designs. This work serves as a capstone for recent developmental efforts that combine traditional survey estimation approaches with the Bayesian modeling paradigm and provides a bridge across the two rich but disparate sub-fields.

stat.ME↗

arXiv2021

Bayesian Pseudo Posterior Mechanism under Asymptotic Differential Privacy

We propose a Bayesian pseudo posterior mechanism to generate record-level synthetic databases equipped with an $(ε,δ)-$ probabilistic differential privacy (pDP) guarantee, where $δ$ denotes the probability that any observed database exceeds $ε$. The pseudo posterior mechanism employs a data record-indexed, risk-based weight vector with weight values $\in [0, 1]$ that surgically downweight the likelihood contributions for high-risk records for model estimation and the generation of record-level synthetic data for public release. The pseudo posterior synthesizer constructs a weight for each data record using the Lipschitz bound for that record under a log-pseudo likelihood utility function that generalizes the exponential mechanism (EM) used to construct a formally private data generating mechanism. By selecting weights to remove likelihood contributions with non-finite log-likelihood values, we guarantee a finite local privacy guarantee for our pseudo posterior mechanism at every sample size. Our results may be applied to \emph{any} synthesizing model envisioned by the data disseminator in a computationally tractable way that only involves estimation of a pseudo posterior distribution for parameters, $θ$, unlike recent approaches that use naturally-bounded utility functions implemented through the EM. We specify mild conditions that guarantee the asymptotic contraction of $δ$ to $0$ over the space of databases. We illustrate our pseudo posterior mechanism on the sensitive family income variable from the Consumer Expenditure Surveys database published by the U.S. Bureau of Labor Statistics. We show that utility is better preserved in the synthetic data for our pseudo posterior mechanism as compared to the EM, both estimated using the same non-private synthesizer, due to our use of targeted downweighting.

stat.ME↗

arXiv2021

Risk-Efficient Bayesian Data Synthesis for Privacy Protection

Statistical agencies utilize models to synthesize respondent-level data for release to the public for privacy protection. In this work, we efficiently induce privacy protection into any Bayesian synthesis model by employing a pseudo likelihood that exponentiates each likelihood contribution by an observation record-indexed weight in [0, 1], defined to be inversely proportional to the identification risk for that record. We start with the marginal probability of identification risk for a record, which is composed as the probability that the identity of the record may be disclosed. Our application to the Consumer Expenditure Surveys (CE) of the U.S. Bureau of Labor Statistics demonstrates that the marginally risk-adjusted synthesizer provides an overall improved privacy protection; however, the identification risks actually increase for some moderate-risk records after risk-adjusted pseudo posterior estimation synthesis due to increased isolation after weighting; a phenomenon we label "whack-a-mole". We proceed to construct a weight for each record from a collection of pairwise identification risk probabilities with other records, where each pairwise probability measures the joint probability of re-identification of the pair of records, which mitigates the whack-a-mole issue and produces a more efficient set of synthetic data with lower risk and higher utility for the CE data.

stat.ME↗