arXiv subjects

Claire Glanois

Publications and source records attributed to Claire Glanois.

12 recordsLinked to original sources

arXiv2024

From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models

Large Language Models (LLMs) have taken the field of AI by storm, but their adoption in the field of Artificial Life (ALife) has been, so far, relatively reserved. In this work we investigate the potential synergies between LLMs and ALife, drawing on a large body of research in the two fields. We explore the potential of LLMs as tools for ALife research, for example, as operators for evolutionary computation or the generation of open-ended environments. Reciprocally, principles of ALife, such as self-organization, collective intelligence and evolvability can provide an opportunity for shaping the development and functionalities of LLMs, leading to more adaptive and responsive models. By investigating this dynamic interplay, the paper aims to inspire innovative crossover approaches for both ALife and LLM research. Along the way, we examine the extent to which LLMs appear to increasingly exhibit properties such as emergence or collective intelligence, expanding beyond their original goal of generating text, and potentially redefining our perception of lifelike intelligence in artificial systems.

cs.NE↗

arXiv2023

MarioGPT: Open-Ended Text2Level Generation through Large Language Models

Procedural Content Generation (PCG) is a technique to generate complex and diverse environments in an automated way. However, while generating content with PCG methods is often straightforward, generating meaningful content that reflects specific intentions and constraints remains challenging. Furthermore, many PCG algorithms lack the ability to generate content in an open-ended manner. Recently, Large Language Models (LLMs) have shown to be incredibly effective in many diverse domains. These trained LLMs can be fine-tuned, re-using information and accelerating training for new tasks. Here, we introduce MarioGPT, a fine-tuned GPT2 model trained to generate tile-based game levels, in our case Super Mario Bros levels. MarioGPT can not only generate diverse levels, but can be text-prompted for controllable level generation, addressing one of the key challenges of current PCG techniques. As far as we know, MarioGPT is the first text-to-level model and combined with novelty search it enables the generation of diverse levels with varying play-style dynamics (i.e. player paths) and the open-ended discovery of an increasingly diverse range of content. Code available at https://github.com/shyamsn97/mario-gpt.

cs.AI↗

arXiv2023

Differentiable Logic Machines

The integration of reasoning, learning, and decision-making is key to build more general artificial intelligence systems. As a step in this direction, we propose a novel neural-logic architecture, called differentiable logic machine (DLM), that can solve both inductive logic programming (ILP) and reinforcement learning (RL) problems, where the solution can be interpreted as a first-order logic program. Our proposition includes several innovations. Firstly, our architecture defines a restricted but expressive continuous relaxation of the space of first-order logic programs by assigning weights to predicates instead of rules, in contrast to most previous neural-logic approaches. Secondly, with this differentiable architecture, we propose several (supervised and RL) training procedures, based on gradient descent, which can recover a fully-interpretable solution (i.e., logic formula). Thirdly, to accelerate RL training, we also design a novel critic architecture that enables actor-critic algorithms. Fourthly, to solve hard problems, we propose an incremental training procedure that can learn a logic program progressively. Compared to state-of-the-art (SOTA) differentiable ILP methods, DLM successfully solves all the considered ILP problems with a higher percentage of successful seeds (up to 3.5$\times$). On RL problems, without requiring an interpretable solution, DLM outperforms other non-interpretable neural-logic RL approaches in terms of rewards (up to 3.9%). When enforcing interpretability, DLM can solve harder RL problems (e.g., Sorting, Path) Moreover, we show that deep logic programs can be learned via incremental supervised training. In addition to this excellent performance, DLM can scale well in terms of memory and computational time, especially during the testing phase where it can deal with much more constants ($>$2$\times$) than SOTA.

cs.AI↗

arXiv2022

HyperNCA: Growing Developmental Networks with Neural Cellular Automata

In contrast to deep reinforcement learning agents, biological neural networks are grown through a self-organized developmental process. Here we propose a new hypernetwork approach to grow artificial neural networks based on neural cellular automata (NCA). Inspired by self-organising systems and information-theoretic approaches to developmental biology, we show that our HyperNCA method can grow neural networks capable of solving common reinforcement learning tasks. Finally, we explore how the same approach can be used to build developmental metamorphosis networks capable of transforming their weights to solve variations of the initial RL task.

cs.NE↗

arXiv2022

A Survey on Interpretable Reinforcement Learning

Although deep reinforcement learning has become a promising machine learning approach for sequential decision-making problems, it is still not mature enough for high-stake domains such as autonomous driving or medical applications. In such contexts, a learned policy needs for instance to be interpretable, so that it can be inspected before any deployment (e.g., for safety and verifiability reasons). This survey provides an overview of various approaches to achieve higher interpretability in reinforcement learning (RL). To that aim, we distinguish interpretability (as a property of a model) and explainability (as a post-hoc operation, with the intervention of a proxy) and discuss them in the context of RL with an emphasis on the former notion. In particular, we argue that interpretable RL may embrace different facets: interpretable inputs, interpretable (transition/reward) models, and interpretable decision-making. Based on this scheme, we summarize and analyze recent work related to interpretable RL with an emphasis on papers published in the past 10 years. We also discuss briefly some related research areas and point to some potential promising research directions.

cs.LG↗

arXiv2021

Neuro-Symbolic Hierarchical Rule Induction

We propose an efficient interpretable neuro-symbolic model to solve Inductive Logic Programming (ILP) problems. In this model, which is built from a set of meta-rules organised in a hierarchical structure, first-order rules are invented by learning embeddings to match facts and body predicates of a meta-rule. To instantiate it, we specifically design an expressive set of generic meta-rules, and demonstrate they generate a consequent fragment of Horn clauses. During training, we inject a controlled \pw{Gumbel} noise to avoid local optima and employ interpretability-regularization term to further guide the convergence to interpretable rules. We empirically validate our model on various tasks (ILP, visual genome, reinforcement learning) against several state-of-the-art methods.

cs.LG↗

arXiv2021

Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning

We consider the problem of learning fair policies in (deep) cooperative multi-agent reinforcement learning (MARL). We formalize it in a principled way as the problem of optimizing a welfare function that explicitly encodes two important aspects of fairness: efficiency and equity. As a solution method, we propose a novel neural network architecture, which is composed of two sub-networks specifically designed for taking into account the two aspects of fairness. In experiments, we demonstrate the importance of the two sub-networks for fair optimization. Our overall approach is general as it can accommodate any (sub)differentiable welfare function. Therefore, it is compatible with various notions of fairness that have been proposed in the literature (e.g., lexicographic maximin, generalized Gini social welfare function, proportional fairness). Our solution method is generic and can be implemented in various MARL settings: centralized training and decentralized execution, or fully decentralized. Finally, we experimentally validate our approach in various domains and show that it can perform much better than previous methods.

cs.LG↗

arXiv2021

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous construction of complex structures from very few starting cells. Recent developments in NCAs lie in the 2D domain, namely reconstructing target images from a single pixel or infinitely growing 2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines, significantly expanding the capabilities of simulated morphogenetic systems. The code for the experiment in this paper can be found at: https://github.com/real-itu/3d-artefacts-nca.

cs.LG↗

arXiv2020

EvoCraft: A New Challenge for Open-Endedness

This paper introduces EvoCraft, a framework for Minecraft designed to study open-ended algorithms. We introduce an API that provides an open-source Python interface for communicating with Minecraft to place and track blocks. In contrast to previous work in Minecraft that focused on learning to play the game, the grand challenge we pose here is to automatically search for increasingly complex artifacts in an open-ended fashion. Compared to other environments used to study open-endedness, Minecraft allows the construction of almost any kind of structure, including actuated machines with circuits and mechanical components. We present initial baseline results in evolving simple Minecraft creations through both interactive and automated evolution. While evolution succeeds when tasked to grow a structure towards a specific target, it is unable to find a solution when rewarded for creating a simple machine that moves. Thus, EvoCraft offers a challenging new environment for automated search methods (such as evolution) to find complex artifacts that we hope will spur the development of more open-ended algorithms. A Python implementation of the EvoCraft framework is available at: https://github.com/real-itu/Evocraft-py.

cs.AI↗

arXiv2016

Periods of the motivic fundamental groupoid of $\boldsymbol{\mathbb{P}^{1} \diagdown \lbrace 0, μ_{N}, \infty \rbrace}$

In this thesis, following F. Brown's point of view, we look at the Hopf algebra structure of motivic cyclotomic multiple zeta values, which are motivic periods of the fundamental groupoid of $\mathbb{P}^{1} \diagdown \lbrace 0, μ_{N}, \infty \rbrace $. By application of a surjective \textit{period} map (which, under Grothendieck's period conjecture, is an isomorphism), we deduce results (such as generating families, identities, etc.) on cyclotomic multiple zeta values, which are complex numbers. The coaction of this Hopf algebra (explicitly given by a combinatorial formula from A. Goncharov and F. Brown's works) is the dual of the action of a so-called \textit{motivic} Galois group on these specific motivic periods. This entire study was actually motivated by the hope of a Galois theory for periods, which should extend the usual Galois theory for algebraic numbers.

math.NT↗

arXiv2016

Unramified Euler sums and Hoffman $\star$ basis

When looking at how periods of $π_{1}^{\mathfrak{m}}(\mathbb{P}^{1}\diagdown \lbrace 0, 1, \infty \rbrace )$, i.e. multiple zeta values, embeds into periods of $π_{1}^{\mathfrak{m}}(\mathbb{P}^{1}\diagdown \lbrace 0, \pm 1, \infty \rbrace )$, i.e. Euler sums, an explicit criteria via the coaction $Δ$ acting on their motivic versions comes out. In this paper, adopting this Galois descent approach, we present a new basis for the space $\mathcal{H}^{1}$ of motivic multiple zeta values via motivic Euler sums. Up to an analytic conjecture, we also prove that the motivic Hoffman star basis $ζ^{\star, \mathfrak{m}} (2^{a_{1}},3,\cdots,3, 2^{a_{p}}, 3, 2^{b})$ is a basis of $\mathcal{H}^{1}$. Under a general motivic identity that we conjecture, these bases are identical. Other examples of unramified ES with alternating patterns of even and odds are also provided.

math.NT↗

arXiv2015

Motivic unipotent fundamental groupoid of $\mathbb{G}_{m} \setminus μ_{N}$ for $N=2,3,4,6,8$ and Galois descents

We study Galois descents for categories of mixed Tate motives over $\mathcal{O}_{N}[1/N]$, for $N\in \left\{2, 3, 4, 8\right\}$ or $\mathcal{O}_{N}$ for $N=6$, with $\mathcal{O}_{N}$ the ring of integers of the $N^{\text{th}}$ cyclotomic field, and construct families of motivic iterated integrals with prescribed properties. In particular this gives a basis of honorary multiple zeta values (linear combinations of iterated integrals at roots of unity $μ_{N}$ which are multiple zeta values). It also gives a new proof, via Goncharov's coproduct, of Deligne's results: the category of mixed Tate motives over $\mathcal{O}_{k_{N}}[1/N]$, for $N\in \left\{2, 3, 4,8\right\}$ is spanned by the motivic fundamental groupoid of $\mathbb{P}^{1}\setminus\left\{0,μ_{N},\infty \right\}$ with an explicit basis. By applying the period map, we obtain a generating family for multiple zeta values relative to $μ_{N}$.

math.NT↗