arXiv subjects

Hawre Jalal

Publications and source records attributed to Hawre Jalal.

8 recordsLinked to original sources

arXiv2024

Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent

Individual-level state-transition microsimulations (iSTMs) have proliferated for economic evaluations in place of cohort state transition models (cSTMs). Probabilistic economic evaluations quantify decision uncertainty and value of information (VOI). Prior studies show that iSTMs provide unbiased estimates of expected incremental net monetary benefits (EINMB), but statistical properties of their estimates of decision uncertainty and VOI are uncharacterized. We compare such iSTMs-produced estimates to corresponding cSTMs. For a 2-alternative decision and normally distributed incremental costs and benefits, we derive analytical expressions for the probability of being cost-effective and the expected value of perfect information (EVPI) for cSTMs and iSTMs, accounting for correlations in incremental outcomes at the population and individual levels. Numerical simulations illustrate our findings and explore relaxation of normality assumptions or having >2 decision alternatives. iSTM estimates of decision uncertainty and VOI are biased but asymptotically consistent (i.e., bias->0 as number of microsimulated individuals->infinity). Decision uncertainty depends on one tail of the INMB distribution (e.g., P(INMB<=0)) which depends on estimated variance (larger with iSTMs given first-order noise). While iSTMs overestimate EVPI, their direction of bias for the probability of being cost-effective is ambiguous. Bias is larger when uncertainties in incremental costs and effects are negatively correlated. While more samples at the population uncertainty level are interchangeable with more microsimulations for estimating EINMB, minimizing iSTM bias in estimating decision uncertainty and VOI depends on sufficient microsimulations. Analysts should account for this when allocating their computational budgets and, at minimum, characterize such bias in their reported results.

q-bio.QM↗

arXiv2024

A Nonparametric Approach for Estimating the Effective Sample Size in Gaussian Approximation of Expected Value of Sample Information

The effective sample size (ESS) measures the informational value of a probability distribution in terms of an equivalent number of study participants. The ESS plays a crucial role in estimating the Expected Value of Sample Information (EVSI) through the Gaussian approximation approach. Despite the significance of ESS, existing ESS estimation methods within the Gaussian approximation framework are either computationally expensive or potentially inaccurate. To address these limitations, we propose a novel approach that estimates the ESS using the summary statistics of generated datasets and nonparametric regression methods. The simulation results suggest that the proposed method provides accurate ESS estimates at a low computational cost, making it an efficient and practical way to quantify the information contained in the probability distribution of a parameter. Overall, determining the ESS can help analysts understand the uncertainty levels in complex prior distributions in the probability analyses of decision models and perform efficient EVSI calculations.

stat.ME↗

arXiv2024

Estimating the EVSI with Gaussian Approximations and Spline-Based Series Methods

Background. The Expected Value of Sample Information (EVSI) measures the expected benefits that could be obtained by collecting additional data. Estimating EVSI using the traditional nested Monte Carlo method is computationally expensive but the recently developed Gaussian approximation (GA) approach can efficiently estimate EVSI across different sample sizes. However, the conventional GA may result in biased EVSI estimates if the decision models are highly nonlinear. This bias may lead to suboptimal study designs when GA is used to optimize the value of different studies. Therefore, we extend the conventional GA approach to improve its performance for nonlinear decision models. Methods. Our method provides accurate EVSI estimates by approximating the conditional benefit based on two steps. First, a Taylor series approximation is applied to estimate the conditional benefit as a function of the conditional moments of the parameters of interest using a spline, which is fitted to the samples of the parameters and the corresponding benefits. Next, the conditional moments of parameters are approximated by the conventional GA and Fisher information. The proposed approach is applied to several data collection exercises involving non-Gaussian parameters and nonlinear decision models. Its performance is compared with the nested Monte Carlo method, the conventional GA approach, and the nonparametric regression-based method for EVSI calculation. Results. The proposed approach provides accurate EVSI estimates across different sample sizes when the parameters of interest are non-Gaussian and the decision models are nonlinear. The computational cost of the proposed method is similar to other novel methods. Conclusions. The proposed approach can estimate EVSI across sample sizes accurately and efficiently, which may support researchers in determining an economically optimal study design using EVSI.

stat.ME↗

arXiv2022

A Tutorial on Time-Dependent Cohort State-Transition Models in R using a Cost-Effectiveness Analysis Example

In an introductory tutorial, we illustrated building cohort state-transition models (cSTMs) in R, where the state transitions probabilities were constant over time. However, in practice, many cSTMs require transitions, rewards, or both to vary over time (time-dependent). This tutorial illustrates adding two types of time-dependency using a previously published cost-effectiveness analysis of multiple strategies as an example. The first is simulation-time dependence, which allows for the transition probabilities to vary as a function of time as measured since the start of the simulation (e.g., varying probability of death as the cohort ages). The second is state-residence time dependence, allowing for history by tracking the time spent in any particular health state using tunnel states. We use these time-dependent cSTMs to conduct cost-effectiveness and probabilistic sensitivity analyses. We also obtain various epidemiological outcomes of interest from the outputs generated from the cSTM, such as survival probability and disease prevalence, often used for model calibration and validation. We present the mathematical notation first, followed by the R code to execute the calculations. The full R code is provided in a public code repository for broader implementation.

stat.CO↗

arXiv2022

An Introductory Tutorial on Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example

Decision models can combine information from different sources to simulate the long-term consequences of alternative strategies in the presence of uncertainty. A cohort state-transition model (cSTM) is a decision model commonly used in medical decision-making to simulate the transitions of a hypothetical cohort among various health states over time. This tutorial focuses on time-independent cSTM, where transition probabilities among health states remain constant over time. We implement time-independent cSTM in R, an open-source mathematical and statistical programming language. We illustrate time-independent cSTMs using a previously published decision model, calculate costs and effectiveness outcomes, conduct a cost-effectiveness analysis of multiple strategies, including a probabilistic sensitivity analysis. We provide open-source code in R to facilitate wider adoption. In a second, more advanced tutorial, we illustrate time-dependent cSTMs.

stat.AP↗

arXiv2020

BayCANN: Streamlining Bayesian Calibration with Artificial Neural Network Metamodeling

Purpose: Bayesian calibration is theoretically superior to standard direct-search algorithm because it can reveal the full joint posterior distribution of the calibrated parameters. However, to date, Bayesian calibration has not been used often in health decision sciences due to practical and computational burdens. In this paper we propose to use artificial neural networks (ANN) as one solution to these limitations. Methods: Bayesian Calibration using Artificial Neural Networks (BayCANN) involves (1) training an ANN metamodel on a sample of model inputs and outputs, and (2) then calibrating the trained ANN metamodel instead of the full model in a probabilistic programming language to obtain the posterior joint distribution of the calibrated parameters. We demonstrate BayCANN by calibrating a natural history model of colorectal cancer to adenoma prevalence and cancer incidence data. In addition, we compare the efficiency and accuracy of BayCANN against performing a Bayesian calibration directly on the simulation model using an incremental mixture importance sampling (IMIS) algorithm. Results: BayCANN was generally more accurate than IMIS in recovering the "true" parameter values. The ratio of the absolute ANN deviation from the truth compared to IMIS for eight out of the nine calibrated parameters were less than one indicating that BayCANN was more accurate than IMIS. In addition, BayCANN took about 15 minutes total compared to the IMIS method which took 80 minutes. Conclusions: In our case study, BayCANN was more accurate than IMIS and was five-folds faster. Because BayCANN does not depend on the structure of the simulation model, it can be adapted to models of various levels of complexity with minor changes to its structure. We provide BayCANN's open-source implementation in R.

stat.ME↗

arXiv2019

Computing the Expected Value of Sample Information Efficiently: Expertise and Skills Required for Four Model-Based Methods

Objectives: Value of information (VOI) analyses can help policy-makers make informed decisions about whether to conduct and how to design future studies. Historically, a computationally expensive method to compute the Expected Value of Sample Information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, four EVSI approximation methods have made such analyses more feasible and accessible. We provide practical recommendations for analysts computing EVSI by evaluating these novel methods. Methods: Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, analyst's expertise and skills, and software required for four recently developed approximation methods. Information was also collected on the strengths and limitations of each approximation method. Results: All four EVSI methods require a decision-analytic model's probabilistic sensitivity analysis (PSA) output. One of the methods also requires the model to be re-run to obtain new PSA outputs for each EVSI estimation. To compute EVSI, analysts must be familiar with at least one of the following skills: advanced regression modeling, likelihood specification, and Bayesian modeling. All methods have different strengths and limitations, e.g., some methods handle evaluation of study designs with more outcomes more efficiently while others quantify uncertainty in EVSI estimates. All methods are programmed in the statistical language R and two of the methods provide online applications. Conclusion: Our paper helps to inform the choice between four efficient EVSI estimation methods, enabling analysts to assess the methods' strengths and limitations and select the most appropriate EVSI method given their situation and skills.

stat.OT↗

arXiv2019

Calculating the Expected Value of Sample Information in Practice: Considerations from Three Case Studies

Investing efficiently in future research to improve policy decisions is an important goal. Expected Value of Sample Information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Therefore, a number of more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared and therefore their relative advantages and disadvantages are not clear. A consortium of EVSI researchers, including the developers of several approximation methods, compared four EVSI methods using three previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared, and the relative advantages and implementation challenges of the methods were highlighted. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. All the evaluated methods gave estimates similar to those given by traditional Monte Carlo, suggesting that EVSI can now be efficiently computed with confidence in realistic examples.

stat.AP↗