arXiv subjects

Daniel Polani

Publications and source records attributed to Daniel Polani.

At least 37 records · Page 2Linked to original sources

arXiv2021

Skilled motor control implies a low entropy of states but a high entropy of actions

The mastery of skills such as playing tennis or balancing an inverted pendulum implies a very accurate control of movements to achieve the task goals. Traditional accounts of skilled action control that focus on either routinization or perceptual control make opposite predictions about the ways we achieve mastery. The notion of routinization emphasizes the decrease of the variance of our actions, whereas the notion of perceptual control emphasizes the decrease of the variance of the states we visit, but not of the actions we execute. Here, we studied how participants managed control tasks of varying levels of complexity, which consisted in controlling inverted pendulums of different lengths. We used information-theoretic measures to compare the predictions of alternative theoretic accounts that focus on routinization and perceptual control, respectively. Our results indicate that the successful performance of the control task strongly correlates with the decrease of state variability and the increase of action variability. As postulated by perceptual control theory, the mastery of skills consists in achieving stable control goals by flexible means.

q-bio.NC↗

arXiv2021

Human Perception of Intrinsically Motivated Autonomy in Human-Robot Interaction

A challenge in using robots in human-inhabited environments is to design behavior that is engaging, yet robust to the perturbations induced by human interaction. Our idea is to imbue the robot with intrinsic motivation (IM) so that it can handle new situations and appears as a genuine social other to humans and thus be of more interest to a human interaction partner. Human-robot interaction (HRI) experiments mainly focus on scripted or teleoperated robots, that mimic characteristics such as IM to control isolated behavior factors. This article presents a "robotologist" study design that allows comparing autonomously generated behaviors with each other, and, for the first time, evaluates the human perception of IM-based generated behavior in robots. We conducted a within-subjects user study (N=24) where participants interacted with a fully autonomous Sphero BB8 robot with different behavioral regimes: one realizing an adaptive, intrinsically motivated behavior and the other being reactive, but not adaptive. The robot and its behaviors are intentionally kept minimal to concentrate on the effect induced by IM. A quantitative analysis of post-interaction questionnaires showed a significantly higher perception of the dimension "Warmth" compared to the reactive baseline behavior. Warmth is considered a primary dimension for social attitude formation in human social cognition. A human perceived as warm (friendly, trustworthy) experiences more positive social interactions.

cs.RO↗

arXiv2021

AvE: Assistance via Empowerment

One difficulty in using artificial agents for human-assistive applications lies in the challenge of accurately assisting with a person's goal(s). Existing methods tend to rely on inferring the human's goal, which is challenging when there are many potential goals or when the set of candidate goals is difficult to identify. We propose a new paradigm for assistance by instead increasing the human's ability to control their environment, and formalize this approach by augmenting reinforcement learning with human empowerment. This task-agnostic objective preserves the person's autonomy and ability to achieve any eventual state. We test our approach against assistance based on goal inference, highlighting scenarios where our method overcomes failure modes stemming from goal ambiguity or misspecification. As existing methods for estimating empowerment in continuous domains are computationally hard, precluding its use in real time learned assistance, we also propose an efficient empowerment-inspired proxy metric. Using this, we are able to successfully demonstrate our method in a shared autonomy user study for a challenging simulated teleoperation task with human-in-the-loop training.

cs.AI↗

arXiv2020

Causal blankets: Theory and algorithmic framework

We introduce a novel framework to identify perception-action loops (PALOs) directly from data based on the principles of computational mechanics. Our approach is based on the notion of causal blanket, which captures sensory and active variables as dynamical sufficient statistics -- i.e. as the "differences that make a difference." Moreover, our theory provides a broadly applicable procedure to construct PALOs that requires neither a steady-state nor Markovian dynamics. Using our theory, we show that every bipartite stochastic process has a causal blanket, but the extent to which this leads to an effective PALO formulation varies depending on the integrated information of the bipartition.

nlin.AO↗

arXiv2019

Information Decomposition based on Cooperative Game Theory

We offer a new approach to the information decomposition problem in information theory: given a 'target' random variable co-distributed with multiple 'source' variables, how can we decompose the mutual information into a sum of non-negative terms that quantify the contributions of each random variable, not only individually but also in combination? We derive our composition from cooperative game theory. It can be seen as assigning a "fair share" of the mutual information to each combination of the source variables. Our decomposition is based on a different lattice from the usual 'partial information decomposition' (PID) approach, and as a consequence our decomposition has a smaller number of terms: it has analogs of the synergy and unique information terms, but lacks terms corresponding to redundancy. Because of this, it is able to obey equivalents of the axioms known as 'local positivity' and 'identity', which cannot be simultaneously satisfied by a PID measure.

cs.IT↗

arXiv2019

Measuring Time with Minimal Clocks

Being able to measure time, whether directly or indirectly, is a significant advantage for an organism. It allows for the timely reaction to regular or predicted events, reducing the pressure for fast processing of sensory input. Thus, clocks are ubiquitous in biology. In the present paper, we consider minimal abstract pure clocks in different configurations and investigate their characteristic dynamics. We are especially interested in optimally time-resolving clocks. Among these, we find fundamentally diametral clock characteristics, such as oscillatory behavior for purely local time measurement or decay-based clocks measuring time periods of a scale global to the problem. We include also sets of independent clocks ("clock bags"), sequential cascades of clocks and composite clocks with controlled dependency. Clock cascades show a "condensation effect" and the composite clock shows various regimes of markedly different dynamics.

q-bio.OT↗

arXiv2019

Bold Hearts Team Description for RoboCup 2019 (Humanoid Kid Size League)

We participated in the RoboCup 2018 competition in Montreal with our newly developed BoldBot based on the Darwin-OP and mostly self-printed custom parts. This paper is about the lessons learnt from that competition and further developments for the RoboCup 2019 competition. Firstly, we briefly introduce the team along with an overview of past achievements. We then present a simple, standalone 2D simulator we use for simplifying the entry for new members with making basic RoboCup concepts quickly accessible. We describe our approach for semantic-segmentation for our vision used in the 2018 competition, which replaced the lookup-table (LUT) implementation we had before. We also discuss the extra structural support we plan to add to the printed parts of the BoldBot and our transition to ROS 2 as our new middleware. Lastly, we will present a collection of open-source contributions of our team.

cs.RO↗

arXiv2018

Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action Loop

Active inference is an ambitious theory that treats perception, inference and action selection of autonomous agents under the heading of a single principle. It suggests biologically plausible explanations for many cognitive phenomena, including consciousness. In active inference, action selection is driven by an objective function that evaluates possible future actions with respect to current, inferred beliefs about the world. Active inference at its core is independent from extrinsic rewards, resulting in a high level of robustness across e.g.\ different environments or agent morphologies. In the literature, paradigms that share this independence have been summarised under the notion of intrinsic motivations. In general and in contrast to active inference, these models of motivation come without a commitment to particular inference and action selection mechanisms. In this article, we study if the inference and action selection machinery of active inference can also be used by alternatives to the originally included intrinsic motivation. The perception-action loop explicitly relates inference and action selection to the environment and agent memory, and is consequently used as foundation for our analysis. We reconstruct the active inference approach, locate the original formulation within, and show how alternative intrinsic motivations can be used while keeping many of the original features intact. Furthermore, we illustrate the connection to universal reinforcement learning by means of our formalism. Active inference research may profit from comparisons of the dynamics induced by alternative intrinsic motivations. Research on intrinsic motivations may profit from an additional way to implement intrinsically motivated agents that also share the biological plausibility of active inference.

cs.AI↗

arXiv2017

Time as it could Be measured in Artificial Living Systems

Being able to measure time, whether directly or indirectly, is a significant advantage for an organism. It permits it to predict regular events, and prepare for them on time. Thus, clocks are ubiquitous in biology. In the present paper, we consider the most minimal abstract pure clocks and investigate their characteristics with respect to their ability to measure time. Amongst other, we find fundamentally diametral clock characteristics, such as oscillatory behaviour for local time measurement or decay-based clocks measuring time periods in scales global to the problem. We include also cascades of independent clocks ("clock bags") and composite clocks with controlled dependency; the latter show various regimes of markedly different dynamics.

q-bio.OT↗

arXiv2017

Action and perception for spatiotemporal patterns

This is a contribution to the formalization of the concept of agents in multivariate Markov chains. Agents are commonly defined as entities that act, perceive, and are goal-directed. In a multivariate Markov chain (e.g. a cellular automaton) the transition matrix completely determines the dynamics. This seems to contradict the possibility of acting entities within such a system. Here we present definitions of actions and perceptions within multivariate Markov chains based on entity-sets. Entity-sets represent a largely independent choice of a set of spatiotemporal patterns that are considered as all the entities within the Markov chain. For example, the entity-set can be chosen according to operational closure conditions or complete specific integration. Importantly, the perception-action loop also induces an entity-set and is a multivariate Markov chain. We then show that our definition of actions leads to non-heteronomy and that of perceptions specialize to the usual concept of perception in the perception-action loop.

cs.AI↗

arXiv2017

Control Capacity of Partially Observable Dynamic Systems in Continuous Time

Stochastic dynamic control systems relate in a prob- abilistic fashion the space of control signals to the space of corresponding future states. Consequently, stochastic dynamic systems can be interpreted as an information channel between the control space and the state space. In this work we study this control-to-state informartion capacity of stochastic dynamic systems in continuous-time, when the states are observed only partially. The control-to-state capacity, known as empowerment, was shown in the past to be useful in solving various Artificial Intelligence & Control benchmarks, and was used to replace problem-specific utilities. The higher the value of empowerment is, the more optional future states an agent may reach by using its controls inside a given time horizon. The contribution of this work is that we derive an efficient solution for computing the control-to-state information capacity for a linear, partially-observed Gaussian dynamic control system in continuous time, and discover new relationships between control-theoretic and information-theoretic properties of dynamic systems. Particularly, using the derived method, we demonstrate that the capacity between the control signal and the system output does not grow without limits with the length of the control signal. This means that only the near-past window of the control signal contributes effectively to the control-to-state capacity, while most of the information beyond this window is irrelevant for the future state of the dynamic system. We show that empowerment depends on a time constant of a dynamic system.

cs.IT↗

arXiv2016

Towards information based spatiotemporal patterns as a foundation for agent representation in dynamical systems

We present some arguments why existing methods for representing agents fall short in applications crucial to artificial life. Using a thought experiment involving a fictitious dynamical systems model of the biosphere we argue that the metabolism, motility, and the concept of counterfactual variation should be compatible with any agent representation in dynamical systems. We then propose an information-theoretic notion of \emph{integrated spatiotemporal patterns} which we believe can serve as the basic building block of an agent definition. We argue that these patterns are capable of solving the problems mentioned before. We also test this in some preliminary experiments.

cs.AI↗

arXiv2016

Cooperation and antagonism in information exchange in a growth scenario with two species

We consider a simple information-theoretic model of communication, in which two species of bacteria have the option of exchanging information about their environment, thereby improving their chances of survival. For this purpose, we model a system consisting of two species whose dynamics in the world are modelled by a bet-hedging strategy. It is well known that such models lend themselves to elegant information-theoretical interpretations by relating their respective long-term growth rate to the information the individual species has about its environment. We are specifically interested in modelling how this dynamics are affected when the species interact cooperatively or in an antagonistic way in a scenario with limited resources. For this purpose, we consider the exchange of environmental information between the two species in the framework of a game. Our results show that a transition from a cooperative to an antagonistic behaviour in a species results as a response to a change in the availability of resources. Species cooperate in abundance of resources, while they behave antagonistically in scarcity.

cs.MA↗

arXiv2015

Informational parasites in code evolution

In a previous study, we considered an information-theoretic model of code evolution. In it, agents obtain information about their (common) environment by the perception of messages of other agents, which is determined by an interaction probability (the structure of the population). For an agent to understand another agent's messages, the former must either know the identity of the latter, or the code producing the messages must be universally interpretable. A universal code, however, introduces a vulnerability: a parasitic entity can take advantage of it. Here, we investigate this problem. In our specific setting, we consider a parasite to be an agent that tries to inflict as much damage as possible in the mutual understanding of the population (i.e. the parasite acts as a disinformation agent). We show that, after introducing a parasite in the population, the former adopts a code such that it captures the information about the environment that is missing in the population. Such agent would be of great value, but only if the rest of the population could understand its messages. However, it is of little use here, since the parasite utilises the most common messages in the population to express different concepts. Now we let the population respond by updating their codes such that, in this arms race, they again maximise their mutual understanding. As a result, there is a code drift in the population where the utilisation of the messages of the parasite is avoided. A consequence of this is that the information that the parasite possesses but the agents lack becomes understandable and readily available.

cs.MA↗

arXiv2015

An informational study of the evolution of codes and of emerging concepts in populations of agents

We consider the problem of the evolution of a code within a structured population of agents. The agents try to maximise their information about their environment by acquiring information from the outputs of other agents in the population. A naive use of information-theoretic methods would assume that every agent knows how to "interpret" the information offered by other agents. However, this assumes that one "knows" which other agents one observes, and thus which code they use. In our model, however, we wish to preclude that: it is not clear which other agents an agent is observing, and the resulting usable information is therefore influenced by the universality of the code used and by which agents an agent is "listening" to. We further investigate whether an agent who does not directly perceive the environment can distinguish states by observing other agents' outputs. For this purpose, we consider a population of different types of agents "talking" about different concepts, and try to extract new ones by considering their outputs only.

cs.MA↗

arXiv2014

Towards designing artificial universes for artificial agents under interaction closure

We are interested in designing artificial universes for artifi- cial agents. We view artificial agents as networks of high- level processes on top of of a low-level detailed-description system. We require that the high-level processes have some intrinsic explanatory power and we introduce an extension of informational closure namely interaction closure to capture this. Then we derive a method to design artificial universes in the form of finite Markov chains which exhibit high-level pro- cesses that satisfy the property of interaction closure. We also investigate control or information transfer which we see as an building block for networks representing artificial agents.

cs.MA↗

arXiv2014

Don't Believe Everything You Hear; Preserving Relevant Information by Discarding Social Information

Integrating information gained by observing others via Social Bayesian Learning can be beneficial for an agent's performance, but can also enable population wide information cascades that perpetuate false beliefs through the agent population. We show how agents can influence the observation network by changing their probability of observing others, and demonstrate the existence of a population-wide equilibrium, where the advantages and disadvantages of the Social Bayesian update are balanced. We also use the formalism of relevant information to illustrate how negative information cascades are characterized by processing increasing amounts of non-relevant information.

cs.MA↗

arXiv2014

Changing the Environment Based on Empowerment as Intrinsic Motivation

One aspect of intelligence is the ability to restructure your own environment so that the world you live in becomes more beneficial to you. In this paper we investigate how the information-theoretic measure of agent empowerment can provide a task-independent, intrinsic motivation to restructure the world. We show how changes in embodiment and in the environment change the resulting behaviour of the agent and the artefacts left in the world. For this purpose, we introduce an approximation of the established empowerment formalism based on sparse sampling, which is simpler and significantly faster to compute for deterministic dynamics. Sparse sampling also introduces a degree of randomness into the decision making process, which turns out to beneficial for some cases. We then utilize the measure to generate agent behaviour for different agent embodiments in a Minecraft-inspired three dimensional block world. The paradigmatic results demonstrate that empowerment can be used as a suitable generic intrinsic motivation to not only generate actions in given static environments, as shown in the past, but also to modify existing environmental conditions. In doing so, the emerging strategies to modify an agent's environment turn out to be meaningful to the specific agent capabilities, i.e., de facto to its embodiment.

cs.AI↗