arXiv subjects

Kevin J. Wilson

Publications and source records attributed to Kevin J. Wilson.

11 recordsLinked to original sources

arXiv2025

Comparing Methods of Expert Elicitation for Treatment Effect or Borrowing Parameters in Standard and Rare Disease Clinical Trials: A Systematic Mapping Study

Expert elicitation is an invaluable tool for gaining insights into the degree of clinical knowledge surrounding parameters of interest when designing, or supplementing trial data when analysing, a clinical trial. Elicitation is considered particularly useful in cases where limited data are available, such as in rare diseases. This study aims to identify methods of expert elicitation and aggregation for treatment effect or borrowing parameters that are used in the design or analysis stages of clinical trials. A comprehensive review of statistical and non-statistical literature was conducted by database searching, and reference list screening of older, relevant literature reviews. The search took place in October 2024 and identified 366 potentially relevant publications. Of these, 126 were selected for full-text review based on review of titles and abstracts, and 41 publications were deemed eligible for inclusion after a full reading. For each included publication, data were extracted on methods of elicitation and aggregation, the types of parameters elicited, the resulting distributions, the number of experts used, and any training provided to experts. Publication characteristics such as contribution type, journal type, and application to the rare disease setting was also noted. A narrative description of the selected publications was produced, detailing 6 unique methods for expert elicitation and 10 unique methods for aggregation. We discuss the most popular methods used across standard and rare disease clinical trials, along with any strengths and limitations. Overall, there is no formal framework for expert elicitation in clinical trials, and more general methods are applied with little consideration into the specific context and trial designs. This review identifies the methodological gaps in current practice, providing a foundation for future development.

stat.AP↗

arXiv2024

On the development of a practical Bayesian optimisation algorithm for expensive experiments and simulations with changing environmental conditions

Experiments in engineering are typically conducted in controlled environments where parameters can be set to any desired value. This assumes that the same applies in a real-world setting -- an assumption that is often incorrect as many experiments are influenced by uncontrollable environmental conditions such as temperature, humidity and wind speed. When optimising such experiments, the focus should lie on finding optimal values conditionally on these uncontrollable variables. This article extends Bayesian optimisation to the optimisation of systems in changing environments that include controllable and uncontrollable parameters. The extension fits a global surrogate model over all controllable and environmental variables but optimises only the controllable parameters conditional on measurements of the uncontrollable variables. The method is validated on two synthetic test functions and the effects of the noise level, the number of the environmental parameters, the parameter fluctuation, the variability of the uncontrollable parameters, and the effective domain size are investigated. ENVBO, the proposed algorithm resulting from this investigation, is applied to a wind farm simulator with eight controllable and one environmental parameter. ENVBO finds solutions for the full domain of the environmental variable that outperforms results from optimisation algorithms that only focus on a fixed environmental value in all but one case while using a fraction of their evaluation budget. This makes the proposed approach very sample-efficient and cost-effective. An off-the-shelf open-source version of ENVBO is available via the NUBO Python package.

cs.LG↗

arXiv2023

Reconciliation of expert priors for quantities and events and application within the probabilistic Delphi method

We consider the problem of aggregating the judgements of a group of experts to form a single prior distribution representing the judgements of the group. We develop a Bayesian hierarchical model to reconcile the judgements of the group of experts based on elicited quantiles for continuous quantities and probabilities for one-off events. Previous Bayesian reconciliation methods have not been used widely, if at all, in contrast to pooling methods and consensus-based approaches. To address this we embed Bayesian reconciliation within the probabilistic Delphi method. The result is to furnish the outcome of the probabilistic Delphi method with a direct probabilistic interpretation, with the resulting prior representing the judgements of the decision maker. We can use the rationales from the Delphi process to group the experts for the hierarchical modelling. We illustrate the approach with applications to studies evaluating erosion in embankment dams and pump failures in a water pumping station, and assess the properties of the approach using the TU Delft database of expert judgement studies. We see that, even using an off-the-shelf implementation of the approach, it out-performs individual experts, equal weighting of experts and the classical method based on the log score.

stat.ME↗

arXiv2023

Hybrid sample size calculations for cluster randomised trials using assurance

Sample size determination for cluster randomised trials (CRTs) is challenging as it requires robust estimation of the intra-cluster correlation coefficient (ICC). Typically, the sample size is chosen to provide a certain level of power to reject the null hypothesis in a hypothesis test. This relies on the minimal clinically important difference (MCID) and estimates for the standard deviation, ICC and possibly the coefficient of variation of the cluster size. Varying these parameters can have a strong effect on the sample size. In particular, it is sensitive to small differences in the ICC. A relevant ICC estimate is often not available, or the available estimate is imprecise. If the ICC used is far from the unknown true value, this can lead to trials which are substantially over- or under-powered. We propose a hybrid approach using Bayesian assurance to find the sample size for a CRT with a frequentist analysis. Assurance is an alternative to power which incorporates uncertainty on parameters through a prior distribution. We suggest specifying prior distributions for the standard deviation, ICC and coefficient of variation of the cluster size, while still utilising the MCID. We illustrate the approach through the design of a CRT in post-stroke incontinence. We show assurance can be used to find a sample size based on an elicited prior distribution for the ICC, when a power calculation discards all information in the prior except a single point estimate. Results show that this approach can avoid misspecifying sample sizes when prior medians for the ICC are very similar but prior distributions exhibit quite different behaviour. Assurance provides an understanding of the probability of success of a trial given an MCID and can be used to produce sample sizes which are robust to parameter uncertainty. This is especially useful when there is difficulty obtaining reliable parameter estimates.

stat.AP↗

arXiv2023

NUBO: A Transparent Python Package for Bayesian Optimization

NUBO, short for Newcastle University Bayesian Optimisation, is a Bayesian optimization framework for the optimization of expensive-to-evaluate black-box functions, such as physical experiments and computer simulators. Bayesian optimization is a costefficient optimization strategy that uses surrogate modelling via Gaussian processes to represent an objective function and acquisition functions to guide the selection of candidate points to approximate the global optimum of the objective function. NUBO itself focuses on transparency and user experience to make Bayesian optimization easily accessible to researchers from all disciplines. Clean and understandable code, precise references, and thorough documentation ensure transparency, while user experience is ensured by a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimization algorithms. NUBO allows users to tailor Bayesian optimization to their specific problem by writing the optimization loop themselves using the provided building blocks. It supports sequential single-point, parallel multi-point, and asynchronous optimization of bounded, constrained, and/or mixed (discrete and continuous) parameter input spaces. Only algorithms and methods that are extensively tested and validated to perform well are included in NUBO. This ensures that the package remains compact and does not overwhelm the user with an unnecessarily large number of options. The package is written in Python but does not require expert knowledge of Python to optimize your simulators and experiments. NUBO is distributed as open-source software under the BSD 3-Clause license.

cs.LG↗

arXiv2022

Bayesian sample size determination for diagnostic accuracy studies

The development of a new diagnostic test ideally follows a sequence of stages which, amongst other aims, evaluate technical performance. This includes an analytical validity study, a diagnostic accuracy study and an interventional clinical utility study. Current approaches to the design and analysis of the diagnostic accuracy study can suffer from prohibitively large sample sizes and interval estimates with undesirable properties. In this paper, we propose a novel Bayesian approach which takes advantage of information available from the analytical validity stage. We utilise assurance to calculate the required sample size based on the target width of a posterior probability interval and can choose to use or disregard the data from the analytical validity study when subsequently inferring measures of test accuracy. Sensitivity analyses are performed to assess the robustness of the proposed sample size to the choice of prior, and prior-data conflict is evaluated by comparing the data to the prior predictive distributions. We illustrate the proposed approach using a motivating real-life application involving a diagnostic test for ventilator associated pneumonia. Finally, we compare the properties of the proposed approach against commonly used alternatives. The results show that by making better use of existing data from earlier studies, the assurance-based approach can not only reduce the required sample size when compared to alternatives, but can also produce more reliable sample sizes for diagnostic accuracy studies.

stat.ME↗

arXiv2021

Multilevel Emulation for Stochastic Computer Models with Application to Large Offshore Wind farms

Renewable energy projects, such as large offshore wind farms, are critical to achieving low-emission targets set by governments. Stochastic computer models allow us to explore future scenarios to aid decision making whilst considering the most relevant uncertainties. Complex stochastic computer models can be prohibitively slow and thus an emulator may be constructed and deployed to allow for efficient computation. We present a novel heteroscedastic Gaussian Process emulator which exploits cheap approximations to a stochastic offshore wind farm simulator. We also conduct a probabilistic sensitivity analysis to understand the influence of key parameters in the wind farm model which will help us to plan a probability elicitation in the future.

stat.ME↗

arXiv2021

A Comparison of Prior Elicitation Aggregation using the Classical Method and SHELF

Subjective Bayesian prior distributions elicited from experts can be aggregated together to form group priors. This paper compares aggregated priors formed by Equal Weight Aggregation, the Classical Method, and the Sheffield Elicitation Framework to each other and individual expert priors, using an expert elicitation carried out for a clinical trial. Aggregation methods and individual expert prior distributions are compared using proper scoring rules to compare the informativeness and calibration of the distributions. The three aggregation methods outperform the individual experts, and the Sheffield Elicitation Framework performs best amongst them.

stat.ME↗

arXiv2020

Uncertainty representation for early phase clinical test evaluations: a case study

In early clinical test evaluations the potential benefits of the introduction of a new technology into the healthcare system are assessed in the challenging situation of limited available empirical data. The aim of these evaluations is to provide additional evidence for the decision maker, who is typically a funder or the company developing the test, to evaluate which technologies should progress to the next stage of evaluation. In this paper we consider the evaluation of a diagnostic test for patients suffering from Chronic Obstructive Pulmonary Disease (COPD). We describe the use of graphical models, prior elicitation and uncertainty analysis to provide the required evidence to allow the test to progress to the next stage of evaluation. We specifically discuss inferring an influence diagram from a care pathway and conducting an elicitation exercise to allow specification of prior distributions over all model parameters. We describe the uncertainty analysis, via Monte Carlo simulation, which allowed us to demonstrate that the potential value of the test was robust to uncertainties. This paper provides a case study illustrating how a careful Bayesian analysis can be used to enhance early clinical test evaluations.

stat.AP↗

arXiv2018

Identifying the effect of public holidays on daily demand for gas

To reduce operational costs, gas distribution networks require accurate forecasts of the demand for gas. Amongst domestic and commercial customers, demand relates primarily to the weather and patterns of life and work. Public holidays have a pronounced effect which often spreads into neighbouring days. We call this spread the "proximity effect". Traditionally, the days over which the proximity effect is felt are pre-specified in fixed windows around each holiday, allowing no uncertainty in their identification. We are motivated by an application to modelling daily gas demand in two large British regions. We introduce a novel model which does not fix the days on which the proximity effect is felt. Our approach uses a four-state, non-homogeneous hidden Markov model, with cyclic dynamics, where the classification of days as public holidays is observed, but the assignment of days as "pre-holiday", "post-holiday" or "normal" is unknown. The number of days to the preceding and succeeding holidays guide transitions between states. We apply Bayesian inference and illustrate the benefit of our modelling approach. A preliminary version of the model is now being used by one of the UK's regional distribution networks.

stat.AP↗

arXiv2016

Bayes linear kinematics in a dynamic Bayesian survival model

Bayes linear kinematics and Bayes linear Bayes graphical models provide an extension of Bayes linear methods so that full conditional updates may be combined with Bayes linear belief adjustment. In this paper we investigate the application of this approach to a more complicated problem: namely survival analysis with time-dependent covariate effects. We use a piecewise-constant hazard function with a prior in which covariate effects are correlated over time. The need for computationally intensive methods is avoided and the relatively simple structure facilitates interpretation. Our approach eliminates the problem of non-commutativity which was observed in earlier work by Gamerman. We apply the technique to data on survival times for leukemia patients.

stat.ME↗