arXiv subjects

Andrew Duncan

Publications and source records attributed to Andrew Duncan.

At least 19 recordsLinked to original sources

arXiv2026

Stallings foldings for rational subsets of automatic groups

Let $G$ be an automatic group with associated regular language $L$. We describe a procedure for constructing an automaton which recognises elements of a given submonoid or rational subset $K$ of $G$. This builds on work of Kharlampovich, Miasnikov and Weil, on the case where $K$ is a subgroup of $G$. Our construction succeeds, after sufficiently many iterations, whenever $K$ satisfies a certain convexity property, which we call $L$-proximity. We show how to test whether the construction is complete in the case that $K$ is a submonoid; we have no such test for the general case of a rational subset $K$. We focus particularly on the case of a surface group $G$ of genus $g>1$, where $L$ is the language of geodesic words in the standard generators. We use small cancellation theory to obtain a method for constructing $L$-recognisable submonoids of $G$.

math.GR↗

arXiv2026

Diffusion Path Samplers via Sequential Monte Carlo

We develop diffusion-based samplers for target distributions known up to a normalising constant. To this end, we rely on the well-known diffusion path that smoothly interpolates between a simple base distribution and the target, popularised by diffusion models. We tackle the score estimation problem by developing an efficient sequential Monte Carlo sampler that evolves auxiliary variables from conditional distributions along the path, providing principled score and density estimates for time-varying distributions. To control the variance of score estimates, we further propose practical control variate schedules that incur minimal overhead. We adapt this general framework to paths induced by the Ornstein-Uhlenbeck (OU) time-reversal process, stochastic interpolants, and diffusion annealed Langevin dynamics, outlining their trade-offs. Finally, we provide theoretical guarantees and empirically demonstrate the effectiveness of our method on several synthetic and real-world datasets.

stat.ML↗

arXiv2025

Conjugate Generalized Bayesian Inference for Discrete Doubly Intractable Problems

Doubly intractable problems occur when both the likelihood and the posterior are available only in unnormalized form, with computationally intractable normalization constants. Bayesian inference then typically requires direct approximation of the posterior through specialized and typically expensive MCMC methods. In this paper, we provide a computationally efficient alternative in the form of a novel generalized Bayesian posterior that allows for conjugate, closed-form or Gibbs-based MCMC inference within the class of exponential family models for discrete data. We derive theoretical guarantees to characterize the asymptotic behavior of the generalized posterior, supporting its use for inference. The method is evaluated on a range of challenging intractable exponential family models, including the Conway-Maxwell-Poisson graphical model of multivariate count data, autoregressive discrete time series models, and Markov random fields such as the Ising and Potts models. The computational gains are significant; in our experiments, the method is between 10 and 6000 times faster than state-of-the-art Bayesian computational methods.

stat.ME↗

arXiv2025

How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding

Large language models trained on clinical text risk exposing sensitive patient information, yet differential privacy (DP) methods often severely degrade the diagnostic accuracy needed for deployment. Despite rapid progress in DP optimisation and text generation, it remains unclear which privacy-preserving strategy actually works best for clinical language tasks. We present the first systematic head-to-head comparison of four training pipelines for automated diagnostic coding from hospital discharge summaries. All pipelines use identical 1B-parameter models and matched privacy budgets to predict ICD-9 codes. At moderate and relaxed privacy budgets ($\varepsilon \in \{4, 6\}$), knowledge distillation from DP-trained teachers outperforms both direct DP-SGD and DP-synthetic data training, recovering up to 63\% of the non-private performance whilst maintaining strong empirical privacy (membership-inference AUC $\approx$ 0.5). These findings expose large differences in the privacy-utility trade-off across architectures and identify knowledge distillation as the most practical route to privacy-preserving clinical NLP.

cs.LG↗

arXiv2025

Uniform-in-time convergence bounds for Persistent Contrastive Divergence Algorithms

We propose a continuous-time formulation of persistent contrastive divergence (PCD) for maximum likelihood estimation (MLE) of unnormalised densities. Our approach expresses PCD as a coupled, multiscale system of stochastic differential equations (SDEs), which perform optimisation of the parameter and sampling of the associated parametrised density, simultaneously. From this novel formulation, we are able to derive explicit bounds for the error between the PCD iterates and the MLE solution for the model parameter. This is made possible by deriving uniform-in-time (UiT) bounds for the difference in moments between the multiscale system and the averaged regime. An efficient implementation of the continuous-time scheme is introduced, leveraging a class of explicit, stable intregators, stochastic orthogonal Runge-Kutta Chebyshev (S-ROCK), for which we provide explicit error estimates in the long-time regime. This leads to a novel method for training energy-based models (EBMs) with explicit error guarantees.

stat.ML↗

arXiv2025

Permutation closure for multiple context-free languages

We prove that the \emph{permutation closure} of a multiple context-free language is multiple context-free, which extends work of Okhotin and Sorokin [LATA 2020] who showed closure under \emph{cyclic shift}, and complements work of Brandst\"adt [1981, RAIRO Inform. Th\'{e}or.] (resp. Brough \emph{et al.} [2016, Discrete Math. Theor. Comput. Sci.]) who showed the same result for regular, context-sensitive, recursively enumerable (resp. EDT0L and ET0L) languages. In contrast to Okhotin and Sorokin who work with grammars, our proof uses restricted tree stack automata due to Denkinger [DLT 2016].

cs.FL↗

arXiv2025

A substitution lemma for multiple context-free languages

We present a necessary condition for an infinite language to be multiple context-free, which we call a Substitution Lemma. We apply it to show a sample selection of languages are not multiple context-free, including the word problem of the group $F_2\times F_2$. We also show that groups with multiple context-free word problem have decidable rational subset membership problem. Our result contrasts with previous work showing that the standard pumping lemma for context-free languages cannot be generalised to multiple context-free languages, and that weak variants of generalised Ogden's lemma do not apply to multiple context-free languages.

cs.FL↗

arXiv2025

Sampling as Bandits: Evaluation-Efficient Design for Black-Box Densities

We propose bandit importance sampling (BIS), a powerful importance sampling framework tailored for settings in which evaluating the target density is computationally expensive. BIS facilitates accurate sampling while minimizing the required number of target-density evaluations. In contrast to adaptive importance sampling, which optimizes a proposal distribution, BIS directly optimizes the set of samples through a sequential selection process driven by multi-armed bandits. BIS serves as a general framework that accommodates user-defined bandit strategies. Theoretically, the weak convergence of the weighted samples, and thus the consistency of the Monte Carlo estimator, is established regardless of the specific strategy employed. In this paper, we present a practical strategy that leverages Gaussian process surrogates to guide sample selection, adapting the principles of Bayesian optimization for sampling. Comprehensive numerical studies demonstrate the superior performance of BIS across multimodal, heavy-tailed distributions, and real-world Bayesian inference tasks involving Markov random fields.

stat.ME↗

arXiv2025

Retrieval-augmented reasoning with lean language models

This technical report details a novel approach to combining reasoning and retrieval augmented generation (RAG) within a single, lean language model architecture. While existing RAG systems typically rely on large-scale models and external APIs, our work addresses the increasing demand for performant and privacy-preserving solutions deployable in resource-constrained or secure environments. Building on recent developments in test-time scaling and small-scale reasoning models, we develop a retrieval augmented conversational agent capable of interpreting complex, domain-specific queries using a lightweight backbone model. Our system integrates a dense retriever with fine-tuned Qwen2.5-Instruct models, using synthetic query generation and reasoning traces derived from frontier models (e.g., DeepSeek-R1) over a curated corpus, in this case, the NHS A-to-Z condition pages. We explore the impact of summarisation-based document compression, synthetic data design, and reasoning-aware fine-tuning on model performance. Evaluation against both non-reasoning and general-purpose lean models demonstrates that our domain-specific fine-tuning approach yields substantial gains in answer accuracy and consistency, approaching frontier-level performance while remaining feasible for local deployment. All implementation details and code are publicly released to support reproducibility and adaptation across domains.

cs.CL↗

arXiv2025

Emerging Paradigms in the Energy Sector: Forecasting and System Control Optimisation

The energy sector is experiencing rapid transformation due to increasing renewable energy integration, decentralisation of power systems, and a heightened focus on efficiency and sustainability. With energy demand becoming increasingly dynamic and generation sources more variable, advanced forecasting and optimisation strategies are crucial for maintaining grid stability, cost-effectiveness, and environmental sustainability. This paper explores emerging paradigms in energy forecasting and management, emphasizing four critical domains: Energy Demand Forecasting integrated with Weather Data, Building Energy Optimisation, Heat Network Optimisation, and Energy Management System (EMS) Optimisation within a System of Systems (SoS) framework. Leveraging machine learning techniques and Model Predictive Control (MPC), the study demonstrates substantial enhancements in energy efficiency across scales -- from individual buildings to complex interconnected energy networks. Weather-informed demand forecasting significantly improves grid resilience and resource allocation strategies. Smart building optimisation integrates predictive analytics to substantially reduce energy consumption without compromising occupant comfort. Optimising CHP-based heat networks achieves cost and carbon savings while adhering to operational and asset constraints. At the systems level, sophisticated EMS optimisation ensures coordinated control of distributed resources, storage solutions, and demand-side flexibility. Through real-world case studies we highlight the potential of AI-driven automation and integrated control solutions in facilitating a resilient, efficient, and sustainable energy future.

cs.ET↗

arXiv2025

Robust and Conjugate Spatio-Temporal Gaussian Processes

State-space formulations allow for Gaussian process (GP) regression with linear-in-time computational cost in spatio-temporal settings, but performance typically suffers in the presence of outliers. In this paper, we adapt and specialise the robust and conjugate GP (RCGP) framework of Altamirano et al. (2024) to the spatio-temporal setting. In doing so, we obtain an outlier-robust spatio-temporal GP with a computational cost comparable to classical spatio-temporal GPs. We also overcome the three main drawbacks of RCGPs: their unreliable performance when the prior mean is chosen poorly, their lack of reliable uncertainty quantification, and the need to carefully select a hyperparameter by hand. We study our method extensively in finance and weather forecasting applications, demonstrating that it provides a reliable approach to spatio-temporal modelling in the presence of outliers.

stat.CO↗

arXiv2025

Expanding on the BRIAR Dataset: A Comprehensive Whole Body Biometric Recognition Resource at Extreme Distances and Real-World Scenarios (Collections 1-4)

The state-of-the-art in biometric recognition algorithms and operational systems has advanced quickly in recent years providing high accuracy and robustness in more challenging collection environments and consumer applications. However, the technology still suffers greatly when applied to non-conventional settings such as those seen when performing identification at extreme distances or from elevated cameras on buildings or mounted to UAVs. This paper summarizes an extension to the largest dataset currently focused on addressing these operational challenges, and describes its composition as well as methodologies of collection, curation, and annotation.

cs.CV↗

arXiv2024

Deep Optimal Sensor Placement for Black Box Stochastic Simulations

Selecting cost-effective optimal sensor configurations for subsequent inference of parameters in black-box stochastic systems faces significant computational barriers. We propose a novel and robust approach, modelling the joint distribution over input parameters and solution with a joint energy-based model, trained on simulation data. Unlike existing simulation-based inference approaches, which must be tied to a specific set of point evaluations, we learn a functional representation of parameters and solution. This is used as a resolution-independent plug-and-play surrogate for the joint distribution, which can be conditioned over any set of points, permitting an efficient approach to sensor placement. We demonstrate the validity of our framework on a variety of stochastic problems, showing that our method provides highly informative sensor locations at a lower computational cost compared to conventional approaches.

stat.ML↗

arXiv2024

Modelling Global Trade with Optimal Transport

Global trade is shaped by a complex mix of factors beyond supply and demand, including tangible variables like transport costs and tariffs, as well as less quantifiable influences such as political and economic relations. Traditionally, economists model trade using gravity models, which rely on explicit covariates that might struggle to capture these subtler drivers of trade. In this work, we employ optimal transport and a deep neural network to learn a time-dependent cost function from data, without imposing a specific functional form. This approach consistently outperforms traditional gravity models in accuracy and has similar performance to three-way gravity models, while providing natural uncertainty quantification. Applying our framework to global food and agricultural trade, we show that the Global South suffered disproportionately from the war in Ukraine's impact on wheat markets. We also analyse the effects of free-trade agreements and trade disputes with China, as well as Brexit's impact on British trade with Europe, uncovering hidden patterns that trade volumes alone cannot reveal.

math.OC↗

arXiv2024

Towards Multilevel Modelling of Train Passing Events on the Staffordshire Bridge

We suggest a multilevel model, to represent aggregate train-passing events from the Staffordshire bridge monitoring system. We formulate a combined model from simple units, representing strain envelopes (of each train passing) for two types of commuter train. The measurements are treated as a longitudinal dataset and represented with a (low-rank approximation) hierarchical Gaussian process. For each unit in the combined model, we encode domain expertise as boundary condition constraints and work towards a general representation of the strain response. Looking forward, this should allow for the simulation of train types that were previously unobserved in the training data. For example, trains with more passengers or freights with a heavier payload. The strain event simulations are valuable since they can inform further experiments (including FEM calibration, fatigue analysis, or design) to test the bridge in hypothesised scenarios.

stat.AP↗

arXiv2023

Meta-models for transfer learning in source localisation

In practice, non-destructive testing (NDT) procedures tend to consider experiments (and their respective models) as distinct, conducted in isolation and associated with independent data. In contrast, this work looks to capture the interdependencies between acoustic emission (AE) experiments (as meta-models) and then use the resulting functions to predict the model hyperparameters for previously unobserved systems. We utilise a Bayesian multilevel approach (similar to deep Gaussian Processes) where a higher level meta-model captures the inter-task relationships. Our key contribution is how knowledge of the experimental campaign can be encoded between tasks as well as within tasks. We present an example of AE time-of-arrival mapping for source localisation, to illustrate how multilevel models naturally lend themselves to representing aggregate systems in engineering. We constrain the meta-model based on domain knowledge, then use the inter-task functions for transfer learning, predicting hyperparameters for models of previously unobserved experiments (for a specific design).

stat.ML↗

arXiv2022

Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Dataset

Face recognition technology has advanced significantly in recent years due largely to the availability of large and increasingly complex training datasets for use in deep learning models. These datasets, however, typically comprise images scraped from news sites or social media platforms and, therefore, have limited utility in more advanced security, forensics, and military applications. These applications require lower resolution, longer ranges, and elevated viewpoints. To meet these critical needs, we collected and curated the first and second subsets of a large multi-modal biometric dataset designed for use in the research and development (R&D) of biometric recognition technologies under extremely challenging conditions. Thus far, the dataset includes more than 350,000 still images and over 1,300 hours of video footage of approximately 1,000 subjects. To collect this data, we used Nikon DSLR cameras, a variety of commercial surveillance cameras, specialized long-rage R&D cameras, and Group 1 and Group 2 UAV platforms. The goal is to support the development of algorithms capable of accurately recognizing people at ranges up to 1,000 m and from high angles of elevation. These advances will include improvements to the state of the art in face recognition and will support new research in the area of whole-body recognition using methods based on gait and anthropometry. This paper describes methods used to collect and curate the dataset, and the dataset's characteristics at the current stage.

cs.CV↗

arXiv2022

Federated Bayesian Computation via Piecewise Deterministic Markov Processes

When performing Bayesian computations in practice, one is often faced with the challenge that the constituent model components and/or the data are only available in a distributed fashion, e.g. due to privacy concerns or sheer volume. While various methods have been proposed for performing posterior inference in such federated settings, these either make very strong assumptions on the data and/or model or otherwise introduce significant bias when the local posteriors are combined to form an approximation of the target posterior. By leveraging recently developed methods for Markov Chain Monte Carlo (MCMC) based on Piecewise Deterministic Markov Processes (PDMPs), we develop a computation -- and communication -- efficient family of posterior inference algorithms (Fed-PDMC) which provides asymptotically exact approximations of the full posterior over a large class of Bayesian models, allowing heterogenous model and data contributions from each client. We show that communication between clients and the server preserves the privacy of the individual data sources by establishing differential privacy guarantees. We quantify the performance of Fed-PDMC over a class of illustrative analytical case-studies and demonstrate its efficacy on a number of synthetic examples along with realistic Bayesian computation benchmarks.

stat.CO↗