arXiv subjects

Ben D. MacArthur

Publications and source records attributed to Ben D. MacArthur.

10 recordsLinked to original sources

arXiv2026

Leakage and Interpretability in Concept-Based Models

Concept-based Models aim to improve interpretability by predicting high-level intermediate concepts, representing a promising approach for deployment in high-risk scenarios. However, they are known to suffer from information leakage, whereby models exploit unintended information encoded within the learned concepts. We introduce an information-theoretic framework to rigorously characterise and quantify leakage, and define two complementary measures: the concepts-task leakage (CTL) and interconcept leakage (ICL) scores. We show that these measures are strongly predictive of model behaviour under interventions and outperform existing alternatives. Using this framework, we identify the primary causes of leakage and, as a case study, analyse how it manifests in Concept Embedding Models, revealing interconcept and alignment leakage in addition to the concepts-task leakage present by design. Finally, we present a set of practical guidelines for designing concept-based models to reduce leakage and ensure interpretability.

cs.LG↗

arXiv2024

Deep Learning as Ricci Flow

Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow - a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of `global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than $1,500$ DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.

cs.LG↗

arXiv2023

Learning from data with structured missingness

Missing data are an unavoidable complication in many machine learning tasks. When data are `missing at random' there exist a range of tools and techniques to deal with the issue. However, as machine learning studies become more ambitious, and seek to learn from ever-larger volumes of heterogeneous data, an increasingly encountered problem arises in which missing values exhibit an association or structure, either explicitly or implicitly. Such `structured missingness' raises a range of challenges that have not yet been systematically addressed, and presents a fundamental hindrance to machine learning at scale. Here, we outline the current literature and propose a set of grand challenges in learning from data with structured missingness.

stat.ML↗

arXiv2021

Geometry and symmetry in biochemical reaction systems

Complex systems of intracellular biochemical reactions have a central role in regulating cell identities and functions. Biochemical reaction systems are typically studied using the language and tools of graph theory. However, graph representations only describe pairwise interactions between molecular species, and so are not well suited to modelling complex sets of reactions that may involve numerous reactants and/or products. Here we make use of a recently-developed hypergraph theory of chemical reactions that naturally allows for higher-order interactions to explore the geometry and quantify functional redundancy in biochemical reactions systems. Our results constitute a general theory of automorphisms for oriented hypergraphs and describe the effect of automorphism group structure on hypergraph Laplacian spectra.

math.CO↗

arXiv2015

Entropy, Ergodicity and Stem Cell Multipotency

Populations of mammalian stem cells commonly exhibit considerable cell-cell variability. However, the functional role of this diversity is unclear. Here, we analyze expression fluctuations of the stem cell surface marker Sca1 in mouse hematopoietic progenitor cells using a simple stochastic model and find that the observed dynamics naturally lie close to a critical state, thereby producing a diverse population that is able to respond rapidly to environmental changes. We propose an information-theoretic interpretation of these results that views cellular multipotency as an instance of maximum entropy statistical inference.

q-bio.CB↗

arXiv2014

Genome disorder and breast cancer susceptibility

Many common diseases have a complex genetic basis in which large numbers of genetic variations combine with environmental and lifestyle factors to determine risk. However, quantifying such polygenic effects and their relationship to disease risk has been challenging. In order to address these difficulties we developed a global measure of the information content of an individual's genome relative to a reference population, which may be used to assess differences in global genome structure between cases and appropriate controls. Informally this measure, which we call relative genome information (RGI), quantifies the relative "disorder" of an individual's genome. In order to test its ability to predict disease risk we used RGI to compare single nucleotide polymorphism genotypes from two independent samples of women with early-onset breast cancer with three independent sets of controls. We found that RGI was significantly elevated in both sets of breast cancer cases in comparison with all three sets of controls, with disease risk rising sharply with RGI (odds ratio greater than 12 for the highest percentile RGI). Furthermore, we found that these differences are not due to associations with common variants at a small number of disease-associated loci, but rather are due to the combined associations of thousands of markers distributed throughout the genome. Our results indicate that the information content of an individual's genome may be used to measure the risk of a complex disease, and suggest that early-onset breast cancer has a strongly polygenic basis.

q-bio.GN↗

arXiv2010

Microdynamics and Criticality of Adaptive Regulatory Networks

We present a model of adaptive regulatory networks consisting of a simple biologically-motivated rewiring procedure coupled to an elementary stability criterion. The resulting networks exhibit a characteristic stationary heavy-tailed degree distribution, show complex structural microdynamics and self-organize to a dynamically critical state. We show analytically that the observed criticality results from the formation and breaking of transient feedback loops during the adaptive process.

physics.soc-ph↗

arXiv2009

Spectral Characteristics of Network Redundancy

Many real-world complex networks contain a significant amount of structural redundancy, in which multiple vertices play identical topological roles. Such redundancy arises naturally from the simple growth processes which form and shape many real-world systems. Since structurally redundant elements may be permuted without altering network structure, redundancy may be formally investigated by examining network automorphism (symmetry) groups. Here, we use a group-theoretic approach to give a complete description of spectral signatures of redundancy in undirected networks. In particular, we describe how a network's automorphism group may be used to directly associate specific eigenvalues and eigenvectors with specific network motifs.

physics.soc-ph↗

arXiv2008

Network Quotients: Structural Skeletons of Complex Systems

A defining feature of many large empirical networks is their intrinsic complexity. However, many networks also contain a large degree of structural repetition. An immediate question then arises: can we characterize essential network complexity while excluding structural redundancy? In this article we utilize inherent network symmetry to collapse all redundant information from a network, resulting in a coarse-graining which we show to carry the essential structural information of the `parent' network. In the context of algebraic combinatorics, this coarse-graining is known as the \emph{quotient}. We systematically explore the theoretical properties of network quotients and summarize key statistics of a variety of `real-world' quotients with respect to those of their parent networks. In particular, we find that quotients can be substantially smaller than their parent networks yet typically preserve various key functional properties such as complexity (heterogeneity and hubs vertices) and communication (diameter and mean geodesic distance), suggesting that quotients constitute the essential structural skeleton of their parent network. We summarize with a discussion of potential uses of quotients in analysis of biological regulatory networks and ways in which using quotients can reduce the computational complexity of network algorithms.

physics.soc-ph↗

arXiv2007

On Automorphism Groups of Networks

We consider the size and structure of the automorphism groups of a variety of empirical `real-world' networks and find that, in contrast to classical random graph models, many real-world networks are richly symmetric. We relate automorphism group structure to network topology and discuss generic forms of symmetry and their origin in real-world networks.

physics.soc-ph↗