arXiv subjects

Michael Biehl

Publications and source records attributed to Michael Biehl.

At least 19 recordsLinked to original sources

arXiv2025

Aligning Generalisation Between Humans and Machines

Recent advances in AI -- including generative approaches -- have resulted in technology that can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals. The responsible use of AI and its participation in human-AI teams increasingly shows the need for AI alignment, that is, to make AI systems act according to our preferences. A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise. In cognitive science, human generalisation commonly involves abstraction and concept learning. In contrast, AI generalisation encompasses out-of-domain generalisation in machine learning, rule-based reasoning in symbolic AI, and abstraction in neurosymbolic AI. In this perspective paper, we combine insights from AI and cognitive science to identify key commonalities and differences across three dimensions: notions of, methods for, and evaluation of generalisation. We map the different conceptualisations of generalisation in AI and cognitive science along these three dimensions and consider their role for alignment in human-AI teaming. This results in interdisciplinary challenges across AI and cognitive science that must be tackled to provide a foundation for effective and cognitively supported alignment in human-AI teaming scenarios.

cs.AI↗

arXiv2025

Discriminative Subspace Emersion from learning feature relevances across different populations

In a given classification task, the accuracy of the learner is often hampered by finiteness of the training set, high-dimensionality of the feature space and severe overlap between classes. In the context of interpretable learners, with (piecewise) linear separation boundaries, these issues can be mitigated by careful construction of optimization procedures and/or estimation of relevant features for the task. However, when the task is shared across two disjoint populations the main interest is shifted towards estimating a set of features that discriminate the most between the two, when performing classification. We propose a new Discriminative Subspace Emersion (DSE) method to extend subspace learning toward a general relevance learning framework. DSE allows us to identify the most relevant features in distinguishing the classification task across two populations, even in cases of high overlap between classes. The proposed methodology is designed to work with multiple sets of labels and is derived in principle without being tied to a specific choice of base learner. Theoretical and empirical investigations over synthetic and real-world datasets indicate that DSE accurately identifies a common subspace for the classification across different populations. This is shown to be true for a surprisingly high degree of overlap between classes.

cs.LG↗

arXiv2024

Iterated Relevance Matrix Analysis (IRMA) for the identification of class-discriminative subspaces

We introduce and investigate the iterated application of Generalized Matrix Learning Vector Quantizaton for the analysis of feature relevances in classification problems, as well as for the construction of class-discriminative subspaces. The suggested Iterated Relevance Matrix Analysis (IRMA) identifies a linear subspace representing the classification specific information of the considered data sets using Generalized Matrix Learning Vector Quantization (GMLVQ). By iteratively determining a new discriminative subspace while projecting out all previously identified ones, a combined subspace carrying all class-specific information can be found. This facilitates a detailed analysis of feature relevances, and enables improved low-dimensional representations and visualizations of labeled data sets. Additionally, the IRMA-based class-discriminative subspace can be used for dimensionality reduction and the training of robust classifiers with potentially improved performance.

cs.LG↗

arXiv2022

Interpretable Models Capable of Handling Systematic Missingness in Imbalanced Classes and Heterogeneous Datasets

Application of interpretable machine learning techniques on medical datasets facilitate early and fast diagnoses, along with getting deeper insight into the data. Furthermore, the transparency of these models increase trust among application domain experts. Medical datasets face common issues such as heterogeneous measurements, imbalanced classes with limited sample size, and missing data, which hinder the straightforward application of machine learning techniques. In this paper we present a family of prototype-based (PB) interpretable models which are capable of handling these issues. The models introduced in this contribution show comparable or superior performance to alternative techniques applicable in such situations. However, unlike ensemble based models, which have to compromise on easy interpretation, the PB models here do not. Moreover we propose a strategy of harnessing the power of ensembles while maintaining the intrinsic interpretability of the PB models, by averaging the model parameter manifolds. All the models were evaluated on a synthetic (publicly available dataset) in addition to detailed analyses of two real-world medical datasets (one publicly available). Results indicated that the models and strategies we introduced addressed the challenges of real-world medical data, while remaining computationally inexpensive and transparent, as well as similar or superior in performance compared to their alternatives.

cs.LG↗

arXiv2021

Supervised Learning in the Presence of Concept Drift: A modelling framework

We present a modelling framework for the investigation of supervised learning in non-stationary environments. Specifically, we model two example types of learning systems: prototype-based Learning Vector Quantization (LVQ) for classification and shallow, layered neural networks for regression tasks. We investigate so-called student teacher scenarios in which the systems are trained from a stream of high-dimensional, labeled data. Properties of the target task are considered to be non-stationary due to drift processes while the training is performed. Different types of concept drift are studied, which affect the density of example inputs only, the target rule itself, or both. By applying methods from statistical physics, we develop a modelling framework for the mathematical analysis of the training dynamics in non-stationary environments. Our results show that standard LVQ algorithms are already suitable for the training in non-stationary environments to a certain extent. However, the application of weight decay as an explicit mechanism of forgetting does not improve the performance under the considered drift processes. Furthermore, we investigate gradient-based training of layered neural networks with sigmoidal activation functions and compare with the use of rectified linear units (ReLU). Our findings show that the sensitivity to concept drift and the effectiveness of weight decay differs significantly between the two types of activation function.

cs.LG↗

arXiv2020

Complex-valued embeddings of generic proximity data

Proximities are at the heart of almost all machine learning methods. If the input data are given as numerical vectors of equal lengths, euclidean distance, or a Hilbertian inner product is frequently used in modeling algorithms. In a more generic view, objects are compared by a (symmetric) similarity or dissimilarity measure, which may not obey particular mathematical properties. This renders many machine learning methods invalid, leading to convergence problems and the loss of guarantees, like generalization bounds. In many cases, the preferred dissimilarity measure is not metric, like the earth mover distance, or the similarity measure may not be a simple inner product in a Hilbert space but in its generalization a Krein space. If the input data are non-vectorial, like text sequences, proximity-based learning is used or ngram embedding techniques can be applied. Standard embeddings lead to the desired fixed-length vector encoding, but are costly and have substantial limitations in preserving the original data's full information. As an information preserving alternative, we propose a complex-valued vector embedding of proximity data. This allows suitable machine learning algorithms to use these fixed-length, complex-valued vectors for further processing. The complex-valued data can serve as an input to complex-valued machine learning algorithms. In particular, we address supervised learning and use extensions of prototype-based learning. The proposed approach is evaluated on a variety of standard benchmarks and shows strong performance compared to traditional techniques in processing non-metric or non-psd proximity data.

cs.LG↗

arXiv2020

Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation

We study layered neural networks of rectified linear units (ReLU) in a modelling framework for stochastic training processes. The comparison with sigmoidal activation functions is in the center of interest. We compute typical learning curves for shallow networks with K hidden units in matching student teacher scenarios. The systems exhibit sudden changes of the generalization performance via the process of hidden unit specialization at critical sizes of the training set. Surprisingly, our results show that the training behavior of ReLU networks is qualitatively different from that of networks with sigmoidal activations. In networks with K >= 3 sigmoidal hidden units, the transition is discontinuous: Specialized network configurations co-exist and compete with states of poor performance even for very large training sets. On the contrary, the use of ReLU activations results in continuous transitions for all K: For large enough training sets, two competing, differently specialized states display similar generalization abilities, which coincide exactly for large networks in the limit K to infinity.

cs.LG↗

arXiv2019

Feature Relevance Determination for Ordinal Regression in the Context of Feature Redundancies and Privileged Information

Advances in machine learning technologies have led to increasingly powerful models in particular in the context of big data. Yet, many application scenarios demand for robustly interpretable models rather than optimum model accuracy; as an example, this is the case if potential biomarkers or causal factors should be discovered based on a set of given measurements. In this contribution, we focus on feature selection paradigms, which enable us to uncover relevant factors of a given regularity based on a sparse model. We focus on the important specific setting of linear ordinal regression, i.e.\ data have to be ranked into one of a finite number of ordered categories by a linear projection. Unlike previous work, we consider the case that features are potentially redundant, such that no unique minimum set of relevant features exists. We aim for an identification of all strongly and all weakly relevant features as well as their type of relevance (strong or weak); we achieve this goal by determining feature relevance bounds, which correspond to the minimum and maximum feature relevance, respectively, if searched over all equivalent models. In addition, we discuss how this setting enables us to substitute some of the features, e.g.\ due to their semantics, and how to extend the framework of feature relevance intervals to the setting of privileged information, i.e.\ potentially relevant information is available for training purposes only, but cannot be used for the prediction itself.

cs.LG↗

arXiv2019

On-line learning dynamics of ReLU neural networks using statistical physics techniques

We introduce exact macroscopic on-line learning dynamics of two-layer neural networks with ReLU units in the form of a system of differential equations, using techniques borrowed from statistical physics. For the first experiments, numerical solutions reveal similar behavior compared to sigmoidal activation researched in earlier work. In these experiments the theoretical results show good correspondence with simulations. In ove-rrealizable and unrealizable learning scenarios, the learning behavior of ReLU networks shows distinctive characteristics compared to sigmoidal networks.

cs.LG↗

arXiv2019

Prototype-based classifiers in the presence of concept drift: A modelling framework

We present a modelling framework for the investigation of prototype-based classifiers in non-stationary environments. Specifically, we study Learning Vector Quantization (LVQ) systems trained from a stream of high-dimensional, clustered data.We consider standard winner-takes-all updates known as LVQ1. Statistical properties of the input data change on the time scale defined by the training process. We apply analytical methods borrowed from statistical physics which have been used earlier for the exact description of learning in stationary environments. The suggested framework facilitates the computation of learning curves in the presence of virtual and real concept drift. Here we focus on timedependent class bias in the training data. First results demonstrate that, while basic LVQ algorithms are suitable for the training in non-stationary environments, weight decay as an explicit mechanism of forgetting does not improve the performance under the considered drift processes.

cs.LG↗

arXiv2019

Galaxy classification: A machine learning analysis of GAMA catalogue data

We present a machine learning analysis of five labelled galaxy catalogues from the Galaxy And Mass Assembly (GAMA): The SersicCatVIKING and SersicCatUKIDSS catalogues containing morphological features, the GaussFitSimple catalogue containing spectroscopic features, the MagPhys catalogue including physical parameters for galaxies, and the Lambdar catalogue, which contains photometric measurements. Extending work previously presented at the ESANN 2018 conference - in an analysis based on Generalized Relevance Matrix Learning Vector Quantization and Random Forests - we find that neither the data from the individual catalogues nor a combined dataset based on all 5 catalogues fully supports the visual-inspection-based galaxy classification scheme employed to categorise the galaxies. In particular, only one class, the Little Blue Spheroids, is consistently separable from the other classes. To aid further insight into the nature of the employed visual-based classification scheme with respect to physical and morphological features, we present the galaxy parameters that are discriminative for the achieved class distinctions.

astro-ph.GA↗

arXiv2019

Feature Relevance Bounds for Ordinal Regression

The increasing occurrence of ordinal data, mainly sociodemographic, led to a renewed research interest in ordinal regression, i.e. the prediction of ordered classes. Besides model accuracy, the interpretation of these models itself is of high relevance, and existing approaches therefore enforce e.g. model sparsity. For high dimensional or highly correlated data, however, this might be misleading due to strong variable dependencies. In this contribution, we aim for an identification of feature relevance bounds which - besides identifying all relevant features - explicitly differentiates between strongly and weakly relevant features.

cs.LG↗

arXiv2012

A Behavioural Perspective on the Early Evolution of Nervous Systems: A Computational Model of Excitable Myoepithelia

How the very first nervous systems evolved remains a fundamental open question. Molecular and genomic techniques have revolutionized our knowledge of the molecular ingredients behind this transition but not yet provided a clear picture of the morphological and tissue changes involved. Here we focus on a behavioural perspective that centres on movement by muscle contraction. Building on the finding that molecules for chemical neural signalling predate multicellular animals, we investigate a gradual evolutionary scenario for nervous systems that consists of two stages: A) Chemically transmission of electrical activity between adjacent cells provided a primitive form of muscle coordination in a contractile epithelial tissue. B) This primitive form of coordination was subsequently improved upon by evolving the axodendritic processes of modern neurons. We use computer simulations to investigate the first stage. The simulations show that chemical transmission across a contractile sheet can indeed produce useful body scale patterns, but only for small-sized animals. For larger animals the noise in chemical neural signalling interferes. Our results imply that a two-stage scenario is a viable approach to nervous system evolution. The first stage could provide an initial behavioural advantage, as well as a clear scaffold for subsequent improvements in behavioural coordination.

q-bio.NC↗

arXiv2011

How to Evaluate Dimensionality Reduction? - Improving the Co-ranking Matrix

The growing number of dimensionality reduction methods available for data visualization has recently inspired the development of quality assessment measures, in order to evaluate the resulting low-dimensional representation independently from a methods' inherent criteria. Several (existing) quality measures can be (re)formulated based on the so-called co-ranking matrix, which subsumes all rank errors (i.e. differences between the ranking of distances from every point to all others, comparing the low-dimensional representation to the original data). The measures are often based on the partioning of the co-ranking matrix into 4 submatrices, divided at the K-th row and column, calculating a weighted combination of the sums of each submatrix. Hence, the evaluation process typically involves plotting a graph over several (or even all possible) settings of the parameter K. Considering simple artificial examples, we argue that this parameter controls two notions at once, that need not necessarily be combined, and that the rectangular shape of submatrices is disadvantageous for an intuitive interpretation of the parameter. We debate that quality measures, as general and flexible evaluation tools, should have parameters with a direct and intuitive interpretation as to which specific error types are tolerated or penalized. Therefore, we propose to replace K with two parameters to control these notions separately, and introduce a differently shaped weighting on the co-ranking matrix. The two new parameters can then directly be interpreted as a threshold up to which rank errors are tolerated, and a threshold up to which the rank-distances are significant for the evaluation. Moreover, we propose a color representation of local quality to visually support the evaluation process for a given mapping, where every point in the mapping is colored according to its local contribution to the overall quality.

cs.LG↗

arXiv2004

Lattice gas models and Kinetic Monte Carlo simulations of epitaxial growth

A brief introduction is given to Kinetic Monte Carlo (KMC) simulations of epitaxial crystal growth. Molecular Beam Epitaxy (MBE) serves as the prototype example for growth far from equilibrium. However, many of the aspects discussed hear would carry over to other techniques as well. A variety of approaches to the modeling and simulation of epitaxial growth has been applied. They range from the detailed quantum mechanics treatment of microscopic processes to the coarse grained picture in terms of stochastic differential equations or other continuum approaches. Here, the focus is on discrete representations such as lattice gas and Solid-On-Solid (SOS) models. The basic ideas of the corresponding KMC methods are presented. Strengths and weaknesses become apparent in the discussion of several levels of simplification that are possible in this context.

cond-mat.mtrl-sci↗

arXiv2004

Off-lattice Kinetic Monte Carlo simulations of strained heteroepitaxial growth

An off-lattice, continuous space Kinetic Monte Carlo (KMC) algorithm is discussed and applied in the investigation of strained heteroepitaxial crystal growth. As a starting point, we study a simplifying (1+1)-dimensional situation with inter-atomic interactions given by simple pair-potentials. The model exhibits the appearance of strain-induced misfit dislocations at a characteristic film thickness. In our simulations we observe a power law dependence of this critical thickness on the lattice misfit, which is in agreement with experimental results for semiconductor compounds. We furthermore investigate the emergence of strain induced multilayer islands or "Dots" upon an adsorbate wetting layer in the so-called Stranski-Krastanov (SK) growth mode. At a characteristic kinetic film thickness, a transition from monolayer to multilayer islands occurs. We discuss the microscopic causes of the SK-transition and its dependence on the model parameters, i.e. lattice misfit, growth rate, and substrate temperature.

cond-mat.stat-mech↗

arXiv2003

Off-lattice Kinetic Monte Carlo simulations of Stranski-Krastanov-like growth

We investigate strained heteroepitaxial crystal growth in the framework of a simplifying (1+1)-dimensional model by use of off-lattice Kinetic Monte Carlo simulations. Our modified Lennard-Jones system displays the so-called Stranski-Krastanov growth mode: initial pseudomorphic growth ends by the sudden appearance of strain induced multilayer islands upon a persisting wetting layer.

cond-mat.stat-mech↗

arXiv2000

A lattice gas model of II-VI(001) semiconductor surfaces

We introduce an anisotropic two-dimensional lattice gas model of metal terminated II-IV(001) seminconductor surfaces. Important properties of this class of materials are represented by effective NN and NNN interactions, which result in the competition of two vacancy structures on the surface. We demonstrate that the experimentally observed c(2x2)-(2x1) transition of the CdTe(001) surface can be understood as a phase transition in thermal equilbrium. The model is studied by means of transfer matrix and Monte Carlo techniques. The analysis shows that the small energy difference of the competing reconstructions determines to a large extent the nature of the different phases. Possible implications for further experimental research are discussed.

cond-mat.stat-mech↗