arXiv subjects

Sebastian Gottwald

Publications and source records attributed to Sebastian Gottwald.

17 recordsLinked to original sources

arXiv2025

Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback

Learning from reward functions and imitation learning of demonstrations are the two principal approaches for training autonomous systems that interact with an environment through action and observation. Both, however, require human specification for each behaviour to be acquired, a problem for long-lived self-adaptive systems whose goals and operating conditions cannot be fully anticipated at design time. Recently, Goal-Conditioned Supervised Learning (GCSL) through self-imitation has been proposed as a self-supervised alternative: by strategically relabelling goals, agents can derive policy insights from their own experiences. Despite its successes, this framework presents two notable limitations: (1) learning exclusively from self-generated experiences can exacerbate the agents' inherent biases; (2) the relabelling strategy allows agents to focus solely on successful outcomes, precluding them from learning from their mistakes. To address these issues, we propose GCSL with Negative Feedback (GCSL-NF), which evaluates each trajectory twice: positively with respect to relabelled goals, and correctively with respect to the goal originally intended. The corrective target comes from a similarity function learned contrastively from trajectory-induced neighbourhood relations, so that neither a reward function nor a geometric distance needs to be specified. Our experiments show that GCSL-NF overcomes limitations imposed by agents' initial biases, increasingly benefits from negative feedback as learning progresses, and matches or surpasses GCSL- and HER-based methods. By reducing reliance on prespecified reward functions, the proposed approach is particularly relevant for self-adaptive autonomous systems, where adaptation objectives may be diverse, changing, or difficult to engineer.

cs.LG↗

arXiv2024

On the Structure of Information

We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random variable, while information corresponds to risk reduction from no (or partial) knowledge to partial knowledge. This applies to any information measure that is based on expected loss minimization, such as Bregman information, with Shannon information and variance as prominent examples. In each case, fundamental properties like the chain rule, non-negativity, and the relationship between information and divergence are preserved. Because partitions form a lattice under refinement, our general treatment reveals how information can be decomposed into redundant, unique, and synergistic contributions, a question important in applications from neuroscience to machine learning, yet one for which existing formulations lack consensus on foundational definitions and can violate basic properties such as the chain rule or non-negativity. Redundancy corresponds to Aumann's common knowledge, synergy to the gap between separately and jointly observed sources, and unique information is necessarily path-dependent, taking different values depending on what is already known. The resulting partial information decomposition is grounded directly in probability theory, avoids treating scalar information quantities as primitive compositional objects, yields non-negative terms by construction, and offers a more fine-grained credit assignment.

cs.IT↗

arXiv2023

Automatic Bat Call Classification using Transformer Networks

Automatically identifying bat species from their echolocation calls is a difficult but important task for monitoring bats and the ecosystem they live in. Major challenges in automatic bat call identification are high call variability, similarities between species, interfering calls and lack of annotated data. Many currently available models suffer from relatively poor performance on real-life data due to being trained on single call datasets and, moreover, are often too slow for real-time classification. Here, we propose a Transformer architecture for multi-label classification with potential applications in real-time classification scenarios. We train our model on synthetically generated multi-species recordings by merging multiple bats calls into a single recording with multiple simultaneous calls. Our approach achieves a single species accuracy of 88.92% (F1-score of 84.23%) and a multi species macro F1-score of 74.40% on our test set. In comparison to three other tools on the independent and publicly available dataset ChiroVox, our model achieves at least 25.82% better accuracy for single species classification and at least 6.9% better macro F1-score for multi species classification.

cs.CV↗

arXiv2022

Thermodynamic fluctuation theorems govern human sensorimotor learning

The application of thermodynamic reasoning in the study of learning systems has a long tradition. Recently, new tools relating perfect thermodynamic adaptation to the adaptation process have been developed. These results, known as fluctuation theorems, have been tested experimentally in several physical scenarios and, moreover, they have been shown to be valid under broad mathematical conditions. Hence, although not experimentally challenged yet, they are presumed to apply to learning systems as well. Here we address this challenge by testing the applicability of fluctuation theorems in learning systems, more specifically, in human sensorimotor learning. In particular, we relate adaptive movement trajectories in a changing visuomotor rotation task to fully adapted steady-state behavior of individual participants. We find that human adaptive behavior in our task is generally consistent with fluctuation theorem predictions and discuss the merits and limitations of the approach.

cond-mat.stat-mech↗

arXiv2022

Majorization requires infinitely many second laws

Majorization is a fundamental model of uncertainty with several applications in areas ranging from thermodynamics to entanglement theory, and constitutes one of the pillars of the resource-theoretic approach to physics. Here, we improve on its relation to measurement apparatuses. In particular, after discussing what the proper notion of second law in this scenario is, we show that, for a sufficiently large state space, any family of entropy-like functions constituting a second law must be countably infinite. Moreover, we provide an analogous result for a variation of majorization known as thermo-majorization which, in fact, does not require any constraint on the state space provided the equilibrium distribution is not uniform. Lastly, we discuss the applicability of our results to molecular diffusion and catalytic majorization. In this regard, we consider a variation of majorization used in plasma physics as a model of molecular diffusion and show that no finite family of entropy-like functions constituting a second law of molecular diffusion exists. Moreover, we show how our results are useful when dealing with a conjecture regarding catalytic majorization (i.e. trumping). In particular, we show that the sort of characterizations of trumping that have been considered before require an infinite family of real-valued functions.

cond-mat.stat-mech↗

arXiv2022

Countability constraints in order-theoretic approaches to computability

Computability on uncountable sets has no standard formalization, unlike that on countable sets, which is given by Turing machines. Some of the approaches to define computability in these sets rely on order-theoretic structures to translate such notions from Turing machines to uncountable spaces. Since these machines are used as a baseline for computability in these approaches, countability restrictions on the ordered structures are fundamental. Here, we show several relations between the usual countability restrictions in order-theoretic theories of computability and some more common order-theoretic countability constraints, like order density properties and functional characterizations of the order structure in terms of multi-utilities. As a result, we show how computability can be introduced in some order structures via countability order density and multi-utility constraints.

math.LO↗

arXiv2022

Computation as uncertainty reduction: a simplified order-theoretic framework

Although there is a somewhat standard formalization of computability on countable sets given by Turing machines, the same cannot be said about uncountable sets. Among the approaches to define computability in these sets, order-theoretic structures have proven to be useful. Here, we discuss the mathematical structure needed to define computability using order-theoretic concepts. In particular, we introduce a more general framework and discuss its limitations compared to the previous one in domain theory. We expose four features in which the stronger requirements in the domain-theoretic structure allow to improve upon the more general framework: computable elements, computable functions, model dependence of computability and complexity theory. Crucially, we show computability of elements in uncountable spaces can be defined in this new setup, and argue why this is not the case for computable functions. Moreover, we show the stronger setup diminishes the dependence of computability on the chosen order-theoretic structure and that, although a suitable complexity theory can be defined in the stronger framework and the more general one posesses a notion of computable elements, there appears to be no proper notion of element complexity in the latter.

cs.LO↗

arXiv2022

The infinite information gap between mathematical and physical representations

Partial orders have been used to model several experimental setups, going from classical thermodynamics and general relativity to the quantum realm with its resource theories. In order to study such experimental setups, one typically characterizes them via a (numerical) representation, that is, a set of real-valued functions. In the context of resource theory, it is customary to use \textbf{mathematical} representations, i.e. a set of \textbf{measurement outcomes} which characterize the achievable transitions within the experimental setup. However, in line with the minimum energy and maximum entropy principles in classical mechanics and thermodynamics, respectively, one would expect an optimization interpretation for a representation to be called \textbf{physical}. More specifically, a physical representation could consist of a set of competing \textbf{optimization principles} such that a transition happens provided they are all optimized by it. Somewhat surprisingly, we show that this distinction can result in an \textbf{infinite information gap}, with some partial orders having mathematical representations that involve a finite amount of information and requiring infinite information to build a physical representation. We connect this phenomenon with well-known resource-theoretic scenarios like majorization, and develop notions of partial order dimension that run in parallel to the representations that we consider. Our results improve on the classification of preordered spaces in terms of real-valued functions.

math.CO↗

arXiv2022

The classification of preordered spaces in terms of monotones: complexity and optimization

The study of complexity and optimization in decision theory involves both partial and complete characterizations of preferences over decision spaces in terms of real-valued monotones. With this motivation, and following the recent introduction of new classes of monotones, like injective monotones or strict monotone multi-utilities, we present the classification of preordered spaces in terms of both the existence and cardinality of real-valued monotones and the cardinality of the quotient space. In particular, we take advantage of a characterization of real-valued monotones in terms of separating families of increasing sets in order to obtain a more complete classification consisting of classes that are strictly different from each other. As a result, we gain new insight into both complexity and optimization, and clarify their interplay in preordered spaces.

math.CO↗

arXiv2022

Jarzyski's equality and Crooks' fluctuation theorem for general Markov chains with application to decision-making systems

We define common thermodynamic concepts purely within the framework of general Markov chains and derive Jarzynski's equality and Crooks' fluctuation theorem in this setup. In particular, we regard the discrete time case that leads to an asymmetry in the definition of work that appears in the usual formulation of Crooks' fluctuation theorem. We show how this asymmetry can be avoided with an additional condition regarding the energy protocol. The general formulation in terms of Markov chains allows transferring the results to other application areas outside of physics. Here, we discuss how this framework can be applied in the context of decision-making. This involves the definition of the relevant quantities, the assumptions that need to be made for the different fluctuation theorems to hold, as well as the consideration of discrete trajectories instead of the continuous trajectories, which are relevant in physics.

cond-mat.stat-mech↗

arXiv2021

Representing preorders with injective monotones

We introduce a new class of real-valued monotones in preordered spaces, injective monotones. We show that the class of preorders for which they exist lies in between the class of preorders with strict monotones and preorders with countable multi-utilities, improving upon the known classification of preordered spaces through real-valued monotones. We extend several well-known results for strict monotones (Richter-Peleg functions) to injective monotones, we provide a construction of injective monotones from countable multi-utilities, and relate injective monotones to classic results concerning Debreu denseness and order separability. Along the way, we connect our results to Shannon entropy and the uncertainty preorder, obtaining new insights into how they are related. In particular, we show how injective montones can be used to generalize some appealing properties of Jaynes' maximum entropy principle, which is considered a basis for statistical inference and serves as a justification for many regularization techniques that appear throughout machine learning and decision theory.

cs.IT↗

arXiv2020

The Two Kinds of Free Energy and the Bayesian Revolution

The concept of free energy has its origins in 19th century thermodynamics, but has recently found its way into the behavioral and neural sciences, where it has been promoted for its wide applicability and has even been suggested as a fundamental principle of understanding intelligent behavior and brain function. We argue that there are essentially two different notions of free energy in current models of intelligent agency, that can both be considered as applications of Bayesian inference to the problem of action selection: one that appears when trading off accuracy and uncertainty based on a general maximum entropy principle, and one that formulates action selection in terms of minimizing an error measure that quantifies deviations of beliefs and policies from given reference models. The first approach provides a normative rule for action selection in the face of model uncertainty or when information processing capabilities are limited. The second approach directly aims to formulate the action selection problem as an inference problem in the context of Bayesian brain theories, also known as Active Inference in the literature. We elucidate the main ideas and discuss critical technical and conceptual issues revolving around these two notions of free energy that both claim to apply at all levels of decision-making, from the high-level deliberation of reasoning down to the low-level information processing of perception.

q-bio.NC↗

arXiv2019

An Information-theoretic On-line Learning Principle for Specialization in Hierarchical Decision-Making Systems

Information-theoretic bounded rationality describes utility-optimizing decision-makers whose limited information-processing capabilities are formalized by information constraints. One of the consequences of bounded rationality is that resource-limited decision-makers can join together to solve decision-making problems that are beyond the capabilities of each individual. Here, we study an information-theoretic principle that drives division of labor and specialization when decision-makers with information constraints are joined together. We devise an on-line learning rule of this principle that learns a partitioning of the problem space such that it can be solved by specialized linear policies. We demonstrate the approach for decision-making problems whose complexity exceeds the capabilities of individual decision-makers, but can be solved by combining the decision-makers optimally. The strength of the model is that it is abstract and principled, yet has direct applications in classification, regression, reinforcement learning and adaptive control.

cs.LG↗

arXiv2019

Bounded rational decision-making from elementary computations that reduce uncertainty

In its most basic form, decision-making can be viewed as a computational process that progressively eliminates alternatives, thereby reducing uncertainty. Such processes are generally costly, meaning that the amount of uncertainty that can be reduced is limited by the amount of available computational resources. Here, we introduce the notion of elementary computation based on a fundamental principle for probability transfers that reduce uncertainty. Elementary computations can be considered as the inverse of Pigou-Dalton transfers applied to probability distributions, closely related to the concepts of majorization, T-transforms, and generalized entropies that induce a preorder on the space of probability distributions. As a consequence we can define resource cost functions that are order-preserving and therefore monotonic with respect to the uncertainty reduction. This leads to a comprehensive notion of decision-making processes with limited resources. Along the way, we prove several new results on majorization theory, as well as on entropy and divergence measures.

cs.IT↗

arXiv2018

Systems of bounded rational agents with information-theoretic constraints

Specialization and hierarchical organization are important features of efficient collaboration in economical, artificial, and biological systems. Here, we investigate the hypothesis that both features can be explained by the fact that each entity of such a system is limited in a certain way. We propose an information-theoretic approach based on a Free Energy principle, in order to computationally analyze systems of bounded rational agents that deal with such limitations optimally. We find that specialization allows to focus on fewer tasks, thus leading to a more efficient execution, but in turn requires coordination in hierarchical structures of specialized experts and coordinating units. Our results suggest that hierarchical architectures of specialized units at lower levels that are coordinated by units at higher levels are optimal, given that each unit's information-processing capability is limited and conforms to constraints on complexity costs.

cs.MA↗

arXiv2018

Bounded Rational Decision-Making with Adaptive Neural Network Priors

Bounded rationality investigates utility-optimizing decision-makers with limited information-processing power. In particular, information theoretic bounded rationality models formalize resource constraints abstractly in terms of relative Shannon information, namely the Kullback-Leibler Divergence between the agents' prior and posterior policy. Between prior and posterior lies an anytime deliberation process that can be instantiated by sample-based evaluations of the utility function through Markov Chain Monte Carlo (MCMC) optimization. The most simple model assumes a fixed prior and can relate abstract information-theoretic processing costs to the number of sample evaluations. However, more advanced models would also address the question of learning, that is how the prior is adapted over time such that generated prior proposals become more efficient. In this work we investigate generative neural networks as priors that are optimized concurrently with anytime sample-based decision-making processes such as MCMC. We evaluate this approach on toy examples.

cs.AI↗

arXiv2017

Two-term spectral asymptotics for the Dirichlet pseudo-relativistic kinetic energy operator on a bounded domain

Continuing the series of works following Weyl's one-term asymptotic formula for the counting function $N(\lambda)=\sum_{n=1}^\infty(\lambda_n{-}\lambda)_-$ of the eigenvalues of the Dirichlet Laplacian and the much later found two-term expansion on domains with highly regular boundary by Ivrii and Melrose, we prove a two-term asymptotic expansion of the $N$-th Ces\`aro mean of the eigenvalues of $\sqrt{-\Delta + m^2} - m$ for $m>0$ with Dirichlet boundary condition on a bounded domain $\Omega\subset\mathbb R^d$ for $d\geq 2$, extending a result by Frank and Geisinger for the fractional Laplacian ($m=0$) and improving upon the small-time asymptotics of the heat trace $Z(t) = \sum_{n=1}^\infty e^{-t \lambda_n}$ by Ba\~nuelos et al. and Park and Song.

math.SP↗