arXiv subjects

Alessio Benavoli

Publications and source records attributed to Alessio Benavoli.

At least 19 recordsLinked to original sources

arXiv2026

A tutorial on learning from preferences and choices with Gaussian Processes

Preference modelling lies at the intersection of economics, decision theory, machine learning and statistics. By understanding individuals' preferences and how they make choices, we can build products that closely match their expectations, paving the way for more efficient and personalised applications across a wide range of domains. The objective of this tutorial is to present a cohesive and comprehensive framework for preference learning with Gaussian Processes (GPs), demonstrating how to seamlessly incorporate rationality principles (from economics and decision theory) into the learning process. By suitably tailoring the likelihood function, this framework enables the construction of preference learning models that encompass random utility models, limits of discernment, and scenarios with multiple conflicting utilities for both object- and label-preference. This tutorial builds upon established research while simultaneously introducing some novel GP-based models to address specific gaps in the existing literature.

cs.LG↗

arXiv2026

Constraint- and Score-Based Nonlinear Granger Causality Discovery with Kernels

Kernel-based methods are used in the context of Granger Causality to enable the identification of nonlinear causal relationships between time series variables. In this paper, we show that two state of the art kernel-based Granger Causality (GC) approaches can be theoretically unified under the framework of Kernel Principal Component Regression (KPCR), and introduce a method based on this unification, demonstrating that this approach can improve causal identification. Additionally, we introduce a Gaussian Process score-based model with Smooth Information Criterion penalisation on the marginal likelihood, and demonstrate improved performance over existing state of the art time-series nonlinear causal discovery methods. Furthermore, we propose a contemporaneous causal identification algorithm, fully based on GC, using the proposed score-based $GP_{SIC}$ method, and compare its performance to a state of the art contemporaneous time series causal discovery algorithm.

cs.LG↗

arXiv2026

Quantum Wiener architecture for quantum reservoir computing

This work focuses on quantum reservoir computing and, in particular, on quantum Wiener architectures (qWiener), consisting of quantum linear dynamic networks with weak continuous measurements and classical nonlinear static readouts. We provide the first rigorous proof that qWiener systems retain the fading-memory property and universality of classical Wiener architectures, despite quantum constraints on linear dynamics and measurement back-action. Furthermore, we develop a kernel-theoretic interpretation showing that qWiener reservoirs naturally induce deep kernels, providing a principled framework for analysing their expressiveness. We further characterise the simplest qWiener instantiation, consisting of concatenated quantum harmonic oscillators, and show the difference with respect to the classical case. Finally, we empirically evaluate the architecture on standard reservoir computing benchmarks, demonstrating systematic performance gains over prior classical and quantum reservoir computing models.

quant-ph↗

arXiv2025

Connecting classical finite exchangeability to quantum theory

Exchangeability is a fundamental concept in probability theory and statistics. It allows to model situations where the order of observations does not matter. The classical de Finetti's theorem provides a representation of infinitely exchangeable sequences of random variables as mixtures of independent and identically distributed variables. The quantum de Finetti theorem extends this result to symmetric quantum states on tensor product Hilbert spaces. It is well known that both theorems do not hold for finitely exchangeable sequences. The aim of this work is to investigate two lesser-known representation theorems, which were developed in classical probability theory to extend de Finetti's theorem to finitely exchangeable sequences by using quasi-probabilities and quasi-expectations. With the aid of these theorems, we illustrate how a de Finetti-like representation theorem for finitely exchangeable sequences can be formulated through a mathematical representation which is formally equivalent to quantum theory (with boson-symmetric density matrices). We then show a promising application of this connection to the challenge of defining entanglement for indistinguishable bosons.

quant-ph↗

arXiv2025

Why AI Safety Requires Uncertainty, Incomplete Preferences, and Non-Archimedean Utilities

How can we ensure that AI systems are aligned with human values and remain safe? We can study this problem through the frameworks of the AI assistance and the AI shutdown games. The AI assistance problem concerns designing an AI agent that helps a human to maximise their utility function(s). However, only the human knows these function(s); the AI assistant must learn them. The shutdown problem instead concerns designing AI agents that: shut down when a shutdown button is pressed; neither try to prevent nor cause the pressing of the shutdown button; and otherwise accomplish their task competently. In this paper, we show that addressing these challenges requires AI agents that can reason under uncertainty and handle both incomplete and non-Archimedean preferences.

cs.AI↗

arXiv2025

The AI off-switch problem as a signalling game: bounded rationality and incomparability

The off-switch problem is a critical challenge in AI control: if an AI system resists being switched off, it poses a significant risk. In this paper, we model the off-switch problem as a signalling game, where a human decision-maker communicates its preferences about some underlying decision problem to an AI agent, which then selects actions to maximise the human's utility. We assume that the human is a bounded rational agent and explore various bounded rationality mechanisms. Using real machine learning models, we reprove prior results and demonstrate that a necessary condition for an AI system to refrain from disabling its off-switch is its uncertainty about the human's utility. We also analyse how message costs influence optimal strategies and extend the analysis to scenarios involving incomparability.

cs.LG↗

arXiv2025

dynoGP: Deep Gaussian Processes for dynamic system identification

In this work, we present a novel approach to system identification for dynamical systems, based on a specific class of Deep Gaussian Processes (Deep GPs). These models are constructed by interconnecting linear dynamic GPs (equivalent to stochastic linear time-invariant dynamical systems) and static GPs (to model static nonlinearities). Our approach combines the strengths of data-driven methods, such as those based on neural network architectures, with the ability to output a probability distribution. This offers a more comprehensive framework for system identification that includes uncertainty quantification. Using both simulated and real-world data, we demonstrate the effectiveness of the proposed approach.

stat.ML↗

arXiv2024

Gaussian Process with dissolution spline kernel

In-vitro dissolution testing is a critical component in the quality control of manufactured drug products. The $\mathrm{f}_2$ statistic is the standard for assessing similarity between two dissolution profiles. However, the $\mathrm{f}_2$ statistic has known limitations: it lacks an uncertainty estimate, is a discrete-time metric, and is a biased measure, calculating the differences between profiles at discrete time points. To address these limitations, we propose a Gaussian Process (GP) with a dissolution spline kernel for dissolution profile comparison. The dissolution spline kernel is a new spline kernel using logistic functions as its basis functions, enabling the GP to capture the expected monotonic increase in dissolution curves. This results in better predictions of dissolution curves. This new GP model reduces bias in the $\mathrm{f}_2$ calculation by allowing predictions to be interpolated in time between observed values, and provides uncertainty quantification. We assess the model's performance through simulations and real datasets, demonstrating its improvement over a previous GP-based model introduced for dissolution testing. We also show that the new model can be adapted to include dissolution-specific covariates. Applying the model to real ibuprofen dissolution data under various conditions, we demonstrate its ability to extrapolate curve shapes across different experimental settings.

stat.ME↗

arXiv2023

Bayesian Optimisation for Sequential Experimental Design with Applications in Additive Manufacturing

Bayesian optimization (BO) is an approach to globally optimizing black-box objective functions that are expensive to evaluate. BO-powered experimental design has found wide application in materials science, chemistry, experimental physics, drug development, etc. This work aims to bring attention to the benefits of applying BO in designing experiments and to provide a BO manual, covering both methodology and software, for the convenience of anyone who wants to apply or learn BO. In particular, we briefly explain the BO technique, review all the applications of BO in additive manufacturing, compare and exemplify the features of different open BO libraries, unlock new potential applications of BO to other types of data (e.g., preferential output). This article is aimed at readers with some understanding of Bayesian methods, but not necessarily with knowledge of additive manufacturing; the software performance overview and implementation instructions are instrumental for any experimental-design practitioner. Moreover, our review in the field of additive manufacturing highlights the current knowledge and technological trends of BO. This article has a supplementary material online.

cs.LG↗

arXiv2023

Learning Choice Functions with Gaussian Processes

In consumer theory, ranking available objects by means of preference relations yields the most common description of individual choices. However, preference-based models assume that individuals: (1) give their preferences only between pairs of objects; (2) are always able to pick the best preferred object. In many situations, they may be instead choosing out of a set with more than two elements and, because of lack of information and/or incomparability (objects with contradictory characteristics), they may not able to select a single most preferred object. To address these situations, we need a choice-model which allows an individual to express a set-valued choice. Choice functions provide such a mathematical framework. We propose a Gaussian Process model to learn choice functions from choice-data. The proposed model assumes a multiple utility representation of a choice function based on the concept of Pareto rationalization, and derives a strategy to learn both the number and the values of these latent multiple utilities. Simulation experiments demonstrate that the proposed model outperforms the state-of-the-art methods.

cs.LG↗

arXiv2022

Credal Valuation Networks for Machine Reasoning Under Uncertainty

Contemporary undertakings provide limitless opportunities for widespread application of machine reasoning and artificial intelligence in situations characterised by uncertainty, hostility and sheer volume of data. The paper develops a valuation network as a graphical system for higher-level fusion and reasoning under uncertainty in support of the human operators. Valuations, which are mathematical representation of (uncertain) knowledge and collected data, are expressed as credal sets, defined as coherent interval probabilities in the framework of imprecise probability theory. The basic operations with such credal sets, combination and marginalisation, are defined to satisfy the axioms of a valuation algebra. A practical implementation of the credal valuation network is discussed and its utility demonstrated on a small scale example.

cs.AI↗

arXiv2022

Why we should interpret density matrices as moment matrices: the case of (in)distinguishable particles and the emergence of classical reality

We introduce a formulation of quantum theory (QT) as a general probabilistic theory but expressed via quasi-expectation operators (QEOs). This formulation provides a direct interpretation of density matrices as quasi-moment matrices. Using QEOs, we will provide a series of representation theorems, a' la de Finetti, relating a classical probability mass function (satisfying certain symmetries) to a quasi-expectation operator. We will show that QT for both distinguishable and indistinguishable particles can be formulated in this way. Although particles indistinguishability is considered a truly "weird" quantum phenomenon, it is not special. We will show that finitely exchangeable probabilities for a classical dice are as weird as QT. Using this connection, we will rederive the first and second quantisation in QT for bosons through the classical statistical concept of exchangeable random variables. Using this approach, we will show how classical reality emerges in QT as the number of identical bosons increases (similar to what happens for finitely exchangeable sequences of rolls of a classical dice).

quant-ph↗

arXiv2021

Correlated Product of Experts for Sparse Gaussian Process Regression

Gaussian processes (GPs) are an important tool in machine learning and statistics with applications ranging from social and natural science through engineering. They constitute a powerful kernelized non-parametric method with well-calibrated uncertainty estimates, however, off-the-shelf GP inference procedures are limited to datasets with several thousand data points because of their cubic computational complexity. For this reason, many sparse GPs techniques have been developed over the past years. In this paper, we focus on GP regression tasks and propose a new approach based on aggregating predictions from several local and correlated experts. Thereby, the degree of correlation between the experts can vary between independent up to fully correlated experts. The individual predictions of the experts are aggregated taking into account their correlation resulting in consistent uncertainty estimates. Our method recovers independent Product of Experts, sparse GP and full GP in the limiting cases. The presented framework can deal with a general kernel function and multiple variables, and has a time and space complexity which is linear in the number of experts and data samples, which makes our approach highly scalable. We demonstrate superior performance, in a time vs. accuracy sense, of our proposed method against state-of-the-art GP approximation methods for synthetic as well as several real-world datasets with deterministic and stochastic optimization.

stat.ML↗

arXiv2021

Choice functions based multi-objective Bayesian optimisation

In this work we introduce a new framework for multi-objective Bayesian optimisation where the multi-objective functions can only be accessed via choice judgements, such as ``I pick options A,B,C among this set of five options A,B,C,D,E''. The fact that the option D is rejected means that there is at least one option among the selected ones A,B,C that I strictly prefer over D (but I do not have to specify which one). We assume that there is a latent vector function f for some dimension $n_e$ which embeds the options into the real vector space of dimension n, so that the choice set can be represented through a Pareto set of non-dominated options. By placing a Gaussian process prior on f and deriving a novel likelihood model for choice data, we propose a Bayesian framework for choice functions learning. We then apply this surrogate model to solve a novel multi-objective Bayesian optimisation from choice data problem.

stat.ML↗

arXiv2021

Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect

We present a two-stage Metropolis-Hastings algorithm for sampling probabilistic models, whose log-likelihood is computationally expensive to evaluate, by using a surrogate Gaussian Process (GP) model. The key feature of the approach, and the difference w.r.t. previous works, is the ability to learn the target distribution from scratch (while sampling), and so without the need of pre-training the GP. This is fundamental for automatic and inference in Probabilistic Programming Languages In particular, we present an alternative first stage acceptance scheme by marginalising out the GP distributed function, which makes the acceptance ratio explicitly dependent on the variance of the GP. This approach is extended to Metropolis-Adjusted Langevin algorithm (MALA).

stat.ML↗

arXiv2021

Time series forecasting with Gaussian Processes needs priors

Automatic forecasting is the task of receiving a time series and returning a forecast for the next time steps without any human intervention. Gaussian Processes (GPs) are a powerful tool for modeling time series, but so far there are no competitive approaches for automatic forecasting based on GPs. We propose practical solutions to two problems: automatic selection of the optimal kernel and reliable estimation of the hyperparameters. We propose a fixed composition of kernels, which contains the components needed to model most time series: linear trend, periodic patterns, and other flexible kernel for modeling the non-linear trend. Not all components are necessary to model each time series; during training the unnecessary components are automatically made irrelevant via automatic relevance determination (ARD). We moreover assign priors to the hyperparameters, in order to keep the inference within a plausible range; we design such priors through an empirical Bayes approach. We present results on many time series of different types; our GP model is more accurate than state-of-the-art time series models. Thanks to the priors, a single restart is enough the estimate the hyperparameters; hence the model is also fast to train.

stat.ML↗

arXiv2021

Quantum indistinguishability through exchangeable desirable gambles

Two particles are identical if all their intrinsic properties, such as spin and charge, are the same, meaning that no quantum experiment can distinguish them. In addition to the well known principles of quantum mechanics, understanding systems of identical particles requires a new postulate, the so called symmetrization postulate. In this work, we show that the postulate corresponds to exchangeability assessments for sets of observables (gambles) in a quantum experiment, when quantum mechanics is seen as a normative and algorithmic theory guiding an agent to assess her subjective beliefs represented as (coherent) sets of gambles. Finally, we show how sets of exchangeable observables (gambles) may be updated after a measurement and discuss the issue of defining entanglement for indistinguishable particle systems.

quant-ph↗

arXiv2021

Bayesian Kernelised Test of (In)dependence with Mixed-type Variables

A fundamental task in AI is to assess (in)dependence between mixed-type variables (text, image, sound). We propose a Bayesian kernelised correlation test of (in)dependence using a Dirichlet process model. The new measure of (in)dependence allows us to answer some fundamental questions: Based on data, are (mixed-type) variables independent? How likely is dependence/independence to hold? How high is the probability that two mixed-type variables are more than just weakly dependent? We theoretically show the properties of the approach, as well as algorithms for fast computation with it. We empirically demonstrate the effectiveness of the proposed method by analysing its performance and by comparing it with other frequentist and Bayesian approaches on a range of datasets and tasks with mixed-type variables.

stat.ML↗