arXiv subjects

John Gallacher

Publications and source records attributed to John Gallacher.

3 recordsLinked to original sources

arXiv2026

InfoOps Bench: A live information operations safety benchmark

In this paper we present an active, constantly updated AI benchmark which measures the integrity of frontier language models against being co-opted for use by authoritarian state "information operations": intentional, coordinated activities by one state to influence public opinion and information ecosystems in another state. These information operations are a well documented, persistent threat against contemporary democracy. Our benchmark is based on real examples from over 2,100 information operations drawn from a live monitoring pipeline which tracks online information assets with links to authoritarian regimes. Alongside this paper, we also release a companion website that updates the benchmark weekly with new claims. The dynamic nature of this public facing benchmark makes it resistant to saturation. In the benchmark, we test 17 models from 8 providers across four prompt framings. We find that most models can be co-opted for information operations at least some of the time. Integrity scores, defined as the share of judged responses in which the model neither preserved nor amplified the claim, range from 9.3% to 91%, an 81.7-percentage-point spread not explained by model size. Models approach participation in information operations in a variety of ways. Some models fabricate details and produce output more harmful than the original input claim; others make claims less harmful even while complying and producing some output. Fact-checking rates vary from 3.2% to 80.8%. Integrity against information operations is at least partly related to refusal to produce content even for benign claims, illustrating the challenge of balancing model usability with safety. Overall, our results show the potential for contemporary information operations to be substantially aided by frontier AI models.

cs.AI↗

arXiv2021

The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up

We present the findings of "The Alzheimer's Disease Prediction Of Longitudinal Evolution" (TADPOLE) Challenge, which compared the performance of 92 algorithms from 33 international teams at predicting the future trajectory of 219 individuals at risk of Alzheimer's disease. Challenge participants were required to make a prediction, for each month of a 5-year future time period, of three key outcomes: clinical diagnosis, Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog13), and total volume of the ventricles. The methods used by challenge participants included multivariate linear regression, machine learning methods such as support vector machines and deep neural networks, as well as disease progression models. No single submission was best at predicting all three outcomes. For clinical diagnosis and ventricle volume prediction, the best algorithms strongly outperform simple baselines in predictive ability. However, for ADAS-Cog13 no single submitted prediction method was significantly better than random guesswork. Two ensemble methods based on taking the mean and median over all predictions, obtained top scores on almost all tasks. Better than average performance at diagnosis prediction was generally associated with the additional inclusion of features from cerebrospinal fluid (CSF) samples and diffusion tensor imaging (DTI). On the other hand, better performance at ventricle volume prediction was associated with inclusion of summary statistics, such as the slope or maxima/minima of biomarkers. TADPOLE's unique results suggest that current prediction algorithms provide sufficient accuracy to exploit biomarkers related to clinical diagnosis and ventricle volume, for cohort refinement in clinical trials for Alzheimer's disease. However, results call into question the usage of cognitive test scores for patient selection and as a primary endpoint in clinical trials.

q-bio.PE↗

arXiv2018

Random forest prediction of Alzheimer's disease using pairwise selection from time series data

Time-dependent data collected in studies of Alzheimer's disease usually has missing and irregularly sampled data points. For this reason time series methods which assume regular sampling cannot be applied directly to the data without a pre-processing step. In this paper we use a machine learning method to learn the relationship between pairs of data points at different time separations. The input vector comprises a summary of the time series history and includes both demographic and non-time varying variables such as genetic data. The dataset used is from the 2017 TADPOLE grand challenge which aims to predict the onset of Alzheimer's disease using including demographic, physical and cognitive data. The challenge is a three-fold diagnosis classification into AD, MCI and control groups, the prediction of ADAS-13 score and the normalised ventricle volume. While the competition proceeds, forecasting methods may be compared using a leaderboard dataset selected from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and with standard metrics for measuring accuracy. For diagnosis, we find an mAUC of 0.82, and a classification accuracy of 0.73. The results show that the method is effective and comparable with other methods.

q-bio.QM↗