arXiv subjects

Paul Gustafson

Publications and source records attributed to Paul Gustafson.

At least 37 records · Page 2Linked to original sources

arXiv2023

Methodological concerns about 'concordance-statistic for benefit' as a measure of discrimination in treatment benefit prediction

Prediction algorithms that quantify the expected benefit of a given treatment conditional on patient characteristics can critically inform medical decisions. Quantifying the performance of treatment benefit prediction algorithms is an active area of research. A recently proposed metric, the concordance statistic for benefit (cfb), evaluates the discriminative ability of a treatment benefit predictor by directly extending the concept of the concordance statistic from a risk model with a binary outcome to a model for treatment benefit. In this work, we scrutinize $cfb$ on multiple fronts. Through numerical examples and theoretical developments, we show that cfb is not a proper scoring rule. We also show that it is sensitive to the unestimable correlation between counterfactual outcomes and to the definition of matched pairs. We argue that measures of statistical dispersion applied to predicted benefits do not suffer from these issues and can be an alternative metric for the discriminatory performance of treatment benefit predictors.

stat.ME↗

arXiv2023

Defining a credible interval is not always possible with "point-null'' priors: A lesser-known correlate of the Jeffreys-Lindley paradox

In many common situations, a Bayesian credible interval will be, given the same data, very similar to a frequentist confidence interval, and researchers will interpret these intervals in a similar fashion. However, no predictable similarity exists when credible intervals are based on model-averaged posteriors whenever one of the two nested models under consideration is a so called ''point-null''. Not only can this model-averaged credible interval be quite different than the frequentist confidence interval, in some cases it may be undefined. This is a lesser-known correlate of the Jeffreys-Lindley paradox and is of particular interest given the popularity of the Bayes factor for testing point-null hypotheses.

math.ST↗

arXiv2023

Automata and one-dimensional TQFTs with defects

This paper explains how any nondeterministic automaton for a regular language $L$ gives rise to a one-dimensional oriented Topological Quantum Field Theory (TQFT) with inner endpoints and zero-dimensional defects labelled by letters of the alphabet for $L$. The TQFT is defined over the Boolean semiring $\mathbb{B}$. Different automata for a fixed language $L$ produce TQFTs that differ by their values on decorated circles, while the values on decorated intervals are described by the language $L$. The language $L$ and the TQFT associated to an automaton can be given a path integral interpretation. In this TQFT the state space of a one-point 0-manifold is a free module over $\mathbb{B}$ with the basis of states of the automaton. Replacing a free module by a finite projective $\mathbb{B}$-module $P$ allows to generalize automata and this type of TQFT to a structure where defects act on open subsets of a finite topological space. Intersection of open subsets induces a multiplication on $P$ allowing to extend the TQFT to a TQFT for one-dimensional foams (oriented graphs with defects modulo a suitable equivalence relation). A linear version of these constructions is also explained, with the Boolean semiring replaced by a commutative ring.

math.QA↗

arXiv2022

Parameter Restrictions for the Sake of Identification: Is there Utility in Asserting that Perhaps a Restriction Holds?

Statistical modeling can involve a tension between assumptions and statistical identification. The law of the observable data may not uniquely determine the value of a target parameter without invoking a key assumption, and, while plausible, this assumption may not be obviously true in the scientific context at hand. Moreover, there are many instances of key assumptions which are untestable, hence we cannot rely on the data to resolve the question of whether the target is legitimately identified. Working in the Bayesian paradigm, we consider the grey zone of situations where a key assumption, in the form of a parameter space restriction, is scientifically reasonable but not incontrovertible for the problem being tackled. Specifically, we investigate statistical properties that ensue if we structure a prior distribution to assert that `maybe' or `perhaps' the assumption holds. Technically this simply devolves to using a mixture prior distribution putting just some prior weight on the assumption, or one of several assumptions, holding. However, while the construct is straightforward, there is very little literature discussing situations where Bayesian model averaging is employed across a mix of fully identified and partially identified models.

stat.ME↗

arXiv2022

Bayes factors and posterior estimation: Two sides of the very same coin

Recently, several researchers have claimed that conclusions obtained from a Bayes factor (or the posterior odds) may contradict those obtained from Bayesian posterior estimation. In this short paper, we wish to point out that no such "incompatibility" exists if one is willing to consistently define one's priors and posteriors. The key for compatibility is that the (implied) prior model odds used for testing are the same as those used for estimation. Our recommendation is simple: If one reports a Bayes factor comparing two models, then one should also report posterior estimates which appropriately acknowledge the uncertainty with regards to which of the two models is correct.

math.ST↗

arXiv2022

From Torus Bundles to Particle-Hole Equivariantization

We continue the program of constructing (pre)modular tensor categories from 3-manifolds first initiated by Cho-Gang-Kim using $M$ theory in physics and then mathematically studied by Cui-Qiu-Wang. An important structure involved is a collection of certain $\text{SL}(2, \mathbb{C})$ characters on a given manifold which serve as the simple object types in the corresponding category. Chern-Simons invariants and adjoint Reidemeister torsions play a key role in the construction, and they are related to topological twists and quantum dimensions, respectively, of simple objects. The modular $S$-matrix is computed from local operators and follows a trial-and-error procedure. It is currently unknown how to produce data beyond the modular $S$- and $T$-matrices. There are also a number of subtleties in the construction which remain to be solved. In this paper, we consider an infinite family of 3-manifolds, that is, torus bundles over the circle. We show that the modular data produced by such manifolds are realized by the $\mathbb{Z}_2$-equivariantization of certain pointed premodular categories. Here the equivariantization is performed for the $\mathbb{Z}_2$-action sending a simple (invertible) object to its inverse, also called the particle-hole symmetry. It is our hope that this extensive class of examples will shed light on how to improve the program to recover the full data of a premodular category.

math.QA↗

arXiv2022

Issues in Implementing Regression Calibration Analyses

Regression calibration is a popular approach for correcting biases in estimated regression parameters when exposure variables are measured with error. This approach involves building a calibration equation to estimate the value of the unknown true exposure given the error-prone measurement and other confounding covariates. The estimated, or calibrated, exposure is then substituted for the true exposure in the health outcome regression model. When used properly, regression calibration can greatly reduce the bias induced by exposure measurement error. Here, we first provide an overview of the statistical framework for regression calibration, specifically discussing how a special type of error, called Berkson error, arises in the estimated exposure. We then present practical issues to consider when applying regression calibration, including: (1) how to develop the calibration equation and which covariates to include; (2) valid ways to calculate standard errors (SE) of estimated regression coefficients; and (3) problems arising if one of the covariates in the calibration model is a mediator of the relationship between the exposure and outcome. Throughout the paper, we provide illustrative examples using data from the Hispanic Community Health Study/Study of Latinos (HCHS/SOL) and simulations. We conclude with recommendations for how to perform regression calibration.

stat.ME↗

arXiv2022

Value of Information Analysis for External Validation of Risk Prediction Models

Background: Before being used to inform patient care, a risk prediction model needs to be validated in a representative sample from the target population. The finite size of the validation sample entails that there is uncertainty with respect to estimates of model performance. We apply value-of-information methodology as a framework to quantify the consequence of such uncertainty in terms of NB. Methods: We define the Expected Value of Perfect Information (EVPI) for model validation as the expected loss in NB due to not confidently knowing which of the alternative decisions confers the highest NB at a given risk threshold. We propose methods for EVPI calculations based on Bayesian or ordinary bootstrapping of NBs, as well as an asymptotic approach supported by the central limit theorem. We conducted brief simulation studies to compare the performance of these methods, and used subsets of data from an international clinical trial for predicting mortality after myocardial infarction as a case study. Results: The three computation methods generated similar EVPI values in simulation studies. In the case study, at the pre-specified threshold of 0.02, the best decision with current information would be to use the model, with an expected incremental NB of 0.0020 over treating all. At this threshold, EVPI was 0.0005 (a relative EVPI of 25%). When scaled to the annual number of heart attacks in the US, this corresponds to a loss of 400 true positives, or extra 19,600 false positives (unnecessary treatments) per year, indicating the value of further model validation. As expected, the validation EVPI generally declined with larger samples. Conclusion: Value-of-information methods can be applied to the NB calculated during external validation of clinical prediction models to provide a decision-theoretic perspective to the consequences of uncertainty.

stat.AP↗

arXiv2022

Closed-Form Solution of the Unit Normal Loss Integral in Two-Dimensions

In Value of Information (VoI) analysis, the unit normal loss integral (UNLI) frequently emerges as a solution for the computation of various VoI metrics. However, one limitation of the UNLI has been that its closed-form solution is available for only one dimension, and thus can be used for comparisons involving only two strategies (where it is applied to the scalar incremental net benefit). We derived a closed-form solution for the two-dimensional UNLI, enabling closed-form VoI calculations for three strategies. We verified the accuracy of this method via simulation studies. A case study based on a three-arm clinical trial was used as an example. VoI methods based on the closed-form solutions for the UNLI can now be extended to three-decision comparisons, taking a fraction of a second to compute and not being subject to Monte Carlo error. An R implementation of this method is provided as part of the predtools package (https://github.com/resplab/predtools/).

stat.CO↗

arXiv2022

re:Linde et al. (2021): The Bayes factor, HDI-ROPE and frequentist equivalence tests can all be reverse engineered -- almost exactly -- from one another

Following an extensive simulation study comparing the operating characteristics of three different procedures used for establishing equivalence (the frequentist "TOST", the Bayesian "HDI-ROPE", and the Bayes factor interval null procedure), Linde et al. (2021) conclude with the recommendation that "researchers rely more on the Bayes factor interval null approach for quantifying evidence for equivalence." We redo the simulation study of Linde et al. (2021) in its entirety but with the different procedures calibrated to have the same predetermined maximum type 1 error rate. Our results suggest that, when calibrated in this way, the Bayes Factor, HDI-ROPE, and frequentist equivalence tests all have similar -- almost exactly -- type 2 error rates. In general any advocating for frequentist testing as better or worse than Bayesian testing in terms of empirical findings seems dubious at best. If one decides on which underlying principle to subscribe to in tackling a given problem, then the method follows naturally. Bearing in mind that each procedure can be reverse-engineered from the others (at least approximately), trying to use empirical performance to argue for one approach over another seems like tilting at windmills.

stat.ME↗

arXiv2021

Uncertainty and Value of Information in Risk Prediction Modeling

Background: Due to the finite size of the development sample, predicted probabilities from a risk prediction model are inevitably uncertain. We apply Value of Information methodology to evaluate the decision-theoretic implications of prediction uncertainty. Methods: Adopting a Bayesian perspective, we extend the definition of the Expected Value of Perfect Information (EVPI) from decision analysis to net benefit calculations in risk prediction. In the context of model development, EVPI is the expected gain in net benefit by using the correct predictions as opposed to predictions from a proposed model. We suggest bootstrap methods for sampling from the posterior distribution of predictions for EVPI calculation using Monte Carlo simulations. In a case study, we used subsets of data of various sizes from a clinical trial for predicting mortality after myocardial infarction to show how EVPI changes with sample size. Results: With a sample size of 1,000 and at the pre-specified threshold of 2% on predicted risks, the gain in net benefit by using the proposed and the correct models were 0.0006 and 0.0011, respectively, resulting in an EVPI of 0.0005 and a relative EVPI of 87%. EVPI was zero only at unrealistically high thresholds (>85%). As expected, EVPI declined with larger samples. We summarize an algorithm for incorporating EVPI calculations into the commonly used bootstrap method for optimism correction. Conclusion: Value of Information methods can be applied to explore decision-theoretic consequences of uncertainty in risk prediction and can complement inferential methods when developing risk prediction models. R code for implementing this method is provided.

stat.AP↗

arXiv2021

Adjusting for misclassification of an exposure in an individual participant data meta-analysis

A common problem in the analysis of multiple data sources, including individual participant data meta-analysis (IPD-MA), is the misclassification of binary variables. Misclassification may lead to biased estimates of model parameters, even when the misclassification is entirely random. We aimed to develop statistical methods that facilitate unbiased estimation of adjusted and unadjusted exposure-outcome associations and between-study heterogeneity in IPD-MA, where the extent and nature of exposure misclassification may vary across studies. We present Bayesian methods that allow misclassification of binary exposure variables to depend on study- and participant-level characteristics. In an example of the differential diagnosis of dengue using two variables, where the gold standard measurement for the exposure variable was unavailable for some studies which only measured a surrogate prone to misclassification, our methods yielded more accurate estimates than analyses naive with regard to misclassification or based on gold standard measurements alone. In a simulation study, the evaluated misclassification model yielded valid estimates of the exposure-outcome association, and was more accurate than analyses restricted to gold standard measurements. Our proposed framework can appropriately account for the presence of binary exposure misclassification in IPD-MA. It requires that some studies supply IPD for the surrogate and gold standard exposure and misclassification is exchangeable across studies conditional on observed covariates (and outcome). The proposed methods are most beneficial when few large studies that measured the gold standard are available, and when misclassification is frequent.

stat.ME↗

arXiv2021

Combining Parametric and Nonparametric Models to Estimate Treatment Effects in Observational Studies

Performing causal inference in observational studies requires we assume confounding variables are correctly adjusted for. G-computation methods are often used in these scenarios, with several recent proposals using Bayesian versions of g-computation. In settings with few confounders, standard models can be employed, however as the number of confounders increase these models become less feasible as there are fewer observations available for each unique combination of confounding variables. In this paper we propose a new model for estimating treatment effects in observational studies that incorporates both parametric and nonparametric outcome models. By conceptually splitting the data, we can combine these models while maintaining a conjugate framework, allowing us to avoid the use of MCMC methods. Approximations using the central limit theorem and random sampling allows our method to be scaled to high dimensional confounders while maintaining computational efficiency. We illustrate the model using carefully constructed simulation studies, as well as compare the computational costs to other benchmark models.

stat.ME↗

arXiv2021

A pasting lemma for Lipschitz functions

We give a necessary and sufficient condition ensuring that any function which is separately Lipschitz on two fixed compact sets is Lipschitz on their union.

math.CA↗

arXiv2021

Hybrid dynamical type theories for navigation

We present a hybrid dynamical type theory equipped with useful primitives for organizing and proving safety of navigational control algorithms. This type theory combines the framework of Fu--Kishida--Selinger for constructing linear dependent type theories from state-parameter fibrations with previous work on categories of hybrid systems under sequential composition. We also define a conjectural embedding of a fragment of linear-time temporal logic within our type theory, with the goal of obtaining interoperability with existing state-of-the-art tools for automatic controller synthesis from formal task specifications. As a case study, we use the type theory to organize and prove safety properties for an obstacle-avoiding navigation algorithm of Arslan--Koditschek as implemented by Vasilopoulos. Finally, we speculate on extensions of the type theory to deal with conjugacies between model and physical spaces, as well as hierarchical template-anchor relationships.

cs.LO↗

arXiv2021

Measurement Error in Meta-Analysis (MEMA) -- a Bayesian framework for continuous outcome data

Ideally, a meta-analysis will summarize data from several unbiased studies. Here we consider the less than ideal situation in which contributing studies may be compromised by measurement error. Measurement error affects every study design, from randomized controlled trials to retrospective observational studies. We outline a flexible Bayesian framework for continuous outcome data which allows one to obtain appropriate point and interval estimates with varying degrees of prior knowledge about the magnitude of the measurement error. We also demonstrate how, if individual-participant data (IPD) are available, the Bayesian meta-analysis model can adjust for multiple participant-level covariates, measured with or without measurement error.

stat.ME↗

arXiv2021

Bayesian adjustment for preferential testing in estimating the COVID-19 infection fatality rate

A key challenge in estimating the infection fatality rate (IFR) -- and its relation with various factors of interest -- is determining the total number of cases. The total number of cases is not known because not everyone is tested, but also, more importantly, because tested individuals are not representative of the population at large. We refer to the phenomenon whereby infected individuals are more likely to be tested than non-infected individuals, as "preferential testing." An open question is whether or not it is possible to reliably estimate the IFR without any specific knowledge about the degree to which the data are biased by preferential testing. In this paper we take a partial identifiability approach, formulating clearly where deliberate prior assumptions can be made and presenting a Bayesian model which pools information from different samples. When the model is fit to European data obtained from seroprevalence studies and national official COVID-19 statistics, we estimate the overall COVID-19 IFR for Europe to be 0.53%, 95% C.I. = [0.39%, 0.69%].

stat.ME↗

arXiv2021

What to make of non-inferiority and equivalence testing with a post-specified margin?

In order to determine whether or not an effect is absent based on a statistical test, the recommended frequentist tool is the equivalence test. Typically, it is expected that an appropriate equivalence margin has been specified before any data are observed. Unfortunately, this can be a difficult task. If the margin is too small, then the test's power will be substantially reduced. If the margin is too large, any claims of equivalence will be meaningless. Moreover, it remains unclear how defining the margin afterwards will bias one's results. In this short article, we consider a series of hypothetical scenarios in which the margin is defined post-hoc or is otherwise considered controversial. We also review a number of relevant, potentially problematic actual studies from clinical trials research, with the aim of motivating a critical discussion as to what is acceptable and desirable in the reporting and interpretation of equivalence tests.

stat.ME↗