arXiv subjects

Manuele Leonelli

Publications and source records attributed to Manuele Leonelli.

At least 37 records · Page 2Linked to original sources

arXiv2024

Learning Staged Trees from Incomplete Data

Staged trees are probabilistic graphical models capable of representing any class of non-symmetric independence via a coloring of its vertices. Several structural learning routines have been defined and implemented to learn staged trees from data, under the frequentist or Bayesian paradigm. They assume a data set has been observed fully and, in practice, observations with missing entries are either dropped or imputed before learning the model. Here, we introduce the first algorithms for staged trees that handle missingness within the learning of the model. To this end, we characterize the likelihood of staged tree models in the presence of missing data and discuss pseudo-likelihoods that approximate it. A structural expectation-maximization algorithm estimating the model directly from the full likelihood is also implemented and evaluated. A computational experiment showcases the performance of the novel learning algorithms, demonstrating that it is feasible to account for different missingness patterns when learning staged trees.

stat.ML↗

arXiv2024

An analysis of factors impacting team strengths in the Australian Football League using time-variant Bradley-Terry models

Australian Rules Football is a field invasion game where two teams attempt to score the highest points to win. Complex machine learning algorithms have been developed to predict match outcomes post-game, but their lack of interpretability hampers an understanding of the factors that affect a team's performance. Using data from the male competition of the Australian Football League, seasons 2015 to 2023, we estimate team strengths and the factors impacting them by fitting flexible Bradley-Terry models. We successfully identify teams significantly stronger or weaker than the average, with stronger teams placing higher in the previous seasons' ladder and leading the activity in the Forward 50 zone, goal shots and scoring over their opponents. Playing at home is confirmed to create an advantage regardless of team strengths. The ability of the model to predict game results in advance is tested, with models accounting for team-specific, time-variant features predicting up to 71.5% of outcomes. Therefore, our approach can provide an interpretable understanding of team strengths and competitive game predictions, making it optimal for data-driven strategies and training.

stat.AP↗

arXiv2024

Staged trees for discrete longitudinal data

In this paper we investigate the use of staged tree models for discrete longitudinal data. Staged trees are a type of probabilistic graphical model for finite sample space processes. They are a natural fit for longitudinal data because a temporal ordering is often implicitly assumed and standard methods can be used for model selection and probability estimation. However, model selection methods perform poorly when the sample size is small relative to the size of the graph and model interpretation is tricky with larger graphs. This is exacerbated by longitudinal data which is characterised by repeated observations. To address these issues we propose two approaches: the longitudinal staged tree with Markov assumptions which makes some initial conditional independence assumptions represented by a directed acyclic graph and marginal longitudinal staged trees which model certain margins of the data.

stat.ME↗

arXiv2024

Robust learning of staged tree models: A case study in evaluating transport services

Staged trees are a relatively recent class of probabilistic graphical models that extend Bayesian networks to formally and graphically account for non-symmetric patterns of dependence. Machine learning algorithms to learn them from data have been implemented in various pieces of software. However, to date, methods to assess the robustness and validity of the learned, non-symmetric relationships are not available. Here, we introduce validation techniques tailored to staged tree models based on non-parametric bootstrap resampling methods and investigate their use in practical applications. In particular, we focus on the evaluation of transport services using large-scale survey data. In these types of applications, data from heterogeneous sources must be collated together. Staged trees provide a natural framework for this integration of data and its analysis. For the thorough evaluation of transport services, we further implement novel what-if sensitivity analyses for staged trees and their visualization using software.

stat.AP↗

arXiv2023

Using Staged Tree Models for Health Data: Investigating Invasive Fungal Infections by Aspergillus and Other Filamentous Fungi

Machine learning models are increasingly used in the medical domain to study the association between risk factors and diseases to support practitioners in predicting health outcomes. In this paper, we showcase the use of machine-learned staged tree models for investigating complex asymmetric dependence structures in health data. Staged trees are a specific class of generative, probabilistic graphical models that formally model asymmetric conditional independence and non-regular sample spaces. An investigation of the risk factors in invasive fungal infections demonstrates the insights staged trees provide to support medical decision-making.

stat.AP↗

arXiv2023

AI and the creative realm: A short review of current and future applications

This study explores the concept of creativity and artificial intelligence (AI) and their recent integration. While AI has traditionally been perceived as incapable of generating new ideas or creating art, the development of more sophisticated AI models and the proliferation of human-computer interaction tools have opened up new possibilities for AI in artistic creation. This study investigates the various applications of AI in a creative context, differentiating between the type of art, language, and algorithms used. It also considers the philosophical implications of AI and creativity, questioning whether consciousness can be researched in machines and AI's potential interests and decision-making capabilities. Overall, we aim to stimulate a reflection on AI's use and ethical implications in creative contexts.

cs.AI↗

arXiv2023

Context-Specific Causal Discovery for Categorical Data Using Staged Trees

Causal discovery algorithms aim at untangling complex causal relationships from data. Here, we study causal discovery and inference methods based on staged tree models, which can represent complex and asymmetric causal relationships between categorical variables. We provide a first graphical representation of the equivalence class of a staged tree, by looking only at a specific subset of its underlying independences. We further define a new pre-metric, inspired by the widely used structural intervention distance, to quantify the closeness between two staged trees in terms of their corresponding causal inference statements. A simulation study highlights the efficacy of staged trees in uncovering complexes, asymmetric causal relationships from data, and real-world data applications illustrate their use in practical causal analysis.

stat.ME↗

arXiv2023

The YODO algorithm: An efficient computational framework for sensitivity analysis in Bayesian networks

Sensitivity analysis measures the influence of a Bayesian network's parameters on a quantity of interest defined by the network, such as the probability of a variable taking a specific value. Various sensitivity measures have been defined to quantify such influence, most commonly some function of the quantity of interest's partial derivative with respect to the network's conditional probabilities. However, computing these measures in large networks with thousands of parameters can become computationally very expensive. We propose an algorithm combining automatic differentiation and exact inference to efficiently calculate the sensitivity measures in a single pass. It first marginalizes the whole network once, using e.g. variable elimination, and then backpropagates this operation to obtain the gradient with respect to all input parameters. Our method can be used for one-way and multi-way sensitivity analysis and the derivation of admissible regions. Simulation studies highlight the efficiency of our algorithm by scaling it to massive networks with up to 100'000 parameters and investigate the feasibility of generic multi-way analyses. Our routines are also showcased over two medium-sized Bayesian networks: the first modeling the country-risks of a humanitarian crisis, the second studying the relationship between the use of technology and the psychological effects of forced social isolation during the COVID-19 pandemic. An implementation of the methods using the popular machine learning library PyTorch is freely available.

stat.ME↗

arXiv2023

Learning and interpreting asymmetry-labeled DAGs: a case study on COVID-19 fear

Bayesian networks are widely used to learn and reason about the dependence structure of discrete variables. However, they are only capable of formally encoding symmetric conditional independence, which in practice is often too strict to hold. Asymmetry-labeled DAGs have been recently proposed to both extend the class of Bayesian networks by relaxing the symmetric assumption of independence and denote the type of dependence existing between the variables of interest. Here, we introduce novel structural learning algorithms for this class of models which, whilst being efficient, allow for a straightforward interpretation of the underlying dependence structure. A comprehensive computational study highlights the efficiency of the algorithms. A real-world data application using data from the Fear of COVID-19 Scale collected in Italy showcases their use in practice.

cs.AI↗

arXiv2022

Staged trees and asymmetry-labeled DAGs

Bayesian networks are a widely-used class of probabilistic graphical models capable of representing symmetric conditional independence between variables of interest using the topology of the underlying graph. For categorical variables, they can be seen as a special case of the much more general class of models called staged trees, which can represent any type of non-symmetric conditional independence. Here we formalize the relationship between these two models and introduce a minimal Bayesian network representation of the staged tree, which can be used to read conditional independences in an intutitive way. A new labeled graph termed asymmetry-labeled directed acyclic graph is defined, whose edges are labeled to denote the type of dependence existing between any two random variables. We also present a novel algorithm to learn staged trees which only enforces a specific subset of non-symmetric independences. Various datasets are used to illustrate the methodology, highlighting the need to construct models which more flexibly encode and represent non-symmetric structures.

stat.ML↗

arXiv2022

A new class of generative classifiers based on staged tree models

Generative models for classification use the joint probability distribution of the class variable and the features to construct a decision rule. Among generative models, Bayesian networks and naive Bayes classifiers are the most commonly used and provide a clear graphical representation of the relationship among all variables. However, these have the disadvantage of highly restricting the type of relationships that could exist, by not allowing for context-specific independences. Here we introduce a new class of generative classifiers, called staged tree classifiers, which formally account for context-specific independence. They are constructed by a partitioning of the vertices of an event tree from which conditional independence can be formally read. The naive staged tree classifier is also defined, which extends the classic naive Bayes classifier whilst retaining the same complexity. An extensive simulation study shows that the classification accuracy of staged tree classifiers is competitive with that of state-of-the-art classifiers and an example showcases their use in practice.

cs.AI↗

arXiv2022

You Only Derive Once (YODO): Automatic Differentiation for Efficient Sensitivity Analysis in Bayesian Networks

Sensitivity analysis measures the influence of a Bayesian network's parameters on a quantity of interest defined by the network, such as the probability of a variable taking a specific value. In particular, the so-called sensitivity value measures the quantity of interest's partial derivative with respect to the network's conditional probabilities. However, finding such values in large networks with thousands of parameters can become computationally very expensive. We propose to use automatic differentiation combined with exact inference to obtain all sensitivity values in a single pass. Our method first marginalizes the whole network once using e.g. variable elimination and then backpropagates this operation to obtain the gradient with respect to all input parameters. We demonstrate our routines by ranking all parameters by importance on a Bayesian network modeling humanitarian crises and disasters, and then show the method's efficiency by scaling it to huge networks with up to 100'000 parameters. An implementation of the methods using the popular machine learning library PyTorch is freely available.

cs.AI↗

arXiv2022

Highly Efficient Structural Learning of Sparse Staged Trees

Several structural learning algorithms for staged tree models, an asymmetric extension of Bayesian networks, have been defined. However, they do not scale efficiently as the number of variables considered increases. Here we introduce the first scalable structural learning algorithm for staged trees, which searches over a space of models where only a small number of dependencies can be imposed. A simulation study as well as a real-world application illustrate our routines and the practical use of such data-learned staged trees.

stat.ML↗

arXiv2022

Structural Learning of Simple Staged Trees

Bayesian networks faithfully represent the symmetric conditional independences existing between the components of a random vector. Staged trees are an extension of Bayesian networks for categorical random vectors whose graph represents non-symmetric conditional independences via vertex coloring. However, since they are based on a tree representation of the sample space, the underlying graph becomes cluttered and difficult to visualize as the number of variables increases. Here we introduce the first structural learning algorithms for the class of simple staged trees, entertaining a compact coalescence of the underlying tree from which non-symmetric independences can be easily read. We show that data-learned simple staged trees often outperform Bayesian networks in model fit and illustrate how the coalesced graph is used to identify non-symmetric conditional independences.

stat.ML↗

arXiv2022

The curved exponential family of a staged tree

Staged tree models are a discrete generalization of Bayesian networks. We show that these form curved exponential families and derive their natural parameters, sufficient statistic, and cumulant-generating function as functions of their graphical representation. We give necessary and sufficient graphical criteria for classifying regular subfamilies and discuss implications for model selection.

math.ST↗

arXiv2021

Global sensitivity analysis in probabilistic graphical models

We show how to apply Sobol's method of global sensitivity analysis to measure the influence exerted by a set of nodes' evidence on a quantity of interest expressed by a Bayesian network. Our method exploits the network structure so as to transform the problem of Sobol index estimation into that of marginalization inference. This way, we can efficiently compute indices for networks where brute-force or Monte Carlo based estimators for variance-based sensitivity analysis would require millions of costly samples. Moreover, our method gives exact results when exact inference is used, and also supports the case of correlated inputs. The proposed algorithm is inspired by the field of tensor networks, and generalizes earlier tensor sensitivity techniques from the acyclic to the cyclic case. We demonstrate the method on three medium to large Bayesian networks that cover the areas of project risk management and reliability engineering.

stat.ML↗

arXiv2021

Sensitivity and robustness analysis in Bayesian networks with the bnmonitor R package

Bayesian networks are a class of models that are widely used for risk assessment of complex operational systems. There are now multiple approaches, as well as implemented software, that guide their construction via data learning or expert elicitation. However, a constructed Bayesian network needs to be validated before it can be used for practical risk assessment. Here, we illustrate the usage of the bnmonitor R package: the first comprehensive software for the validation of a Bayesian network. An applied data analysis using bnmonitor is carried out over a medical dataset to illustrate the use of its wide array of functions.

stat.ME↗

arXiv2021

A geometric characterisation of sensitivity analysis in monomial models

Sensitivity analysis in probabilistic discrete graphical models is usually conducted by varying one probability value at a time and observing how this affects output probabilities of interest. When one probability is varied then others are proportionally covaried to respect the sum-to-one condition of probability laws. The choice of proportional covariation is justified by a variety of optimality conditions, under which the original and the varied distributions are as close as possible under different measures of closeness. For variations of more than one parameter at a time proportional covariation is justified in some special cases only. In this work, for the large class of discrete statistical models entertaining a regular monomial parametrisation, we demonstrate the optimality of newly defined proportional multi-way schemes with respect to an optimality criterion based on the notion of I-divergence. We demonstrate that there are varying parameters choices for which proportional covariation is not optimal and identify the sub-family of model distributions where the distance between the original distribution and the one where probabilities are covaried proportionally is minimum. This is shown by adopting a new formal, geometric characterization of sensitivity analysis in monomial models, which include a wide array of probabilistic graphical models. We also demonstrate the optimality of proportional covariation for multi-way analyses in Naive Bayes classifiers.

math.ST↗