Searcharxiv⌕ Search

arXiv subjects

Isabeau Prémont-Schwarz

Publications and source records attributed to Isabeau Prémont-Schwarz.

17 recordsLinked to original sources
arXiv2026

TPR-Attention for Combinatorial Generalization

Systematic generalization remains a significant challenge in deep learning. In particular, combinatorial generalization - generalizing to new configurations of known factors of variation - is effortless for humans but difficult for standard neural architectures that rely on statistical correlations rather than explicit structural representations. We introduce a new architectural component that embeds structured inductive bias into deep learning: an attention mechanism operating over tensor-product representations (TPRs). Through controlled experiments on compositional tasks, we show that this TPR-attention mechanism outperforms existing architectural components in combinatorial generalization. These results highlight the value of integrating explicit compositional structure into neural attention and point toward a promising path for models capable of systematic generalization.

cs.LG↗
arXiv2024

SNAP: Stopping Catastrophic Forgetting in Hebbian Learning with Sigmoidal Neuronal Adaptive Plasticity

Artificial Neural Networks (ANNs) suffer from catastrophic forgetting, where the learning of new tasks causes the catastrophic forgetting of old tasks. Existing Machine Learning (ML) algorithms, including those using Stochastic Gradient Descent (SGD) and Hebbian Learning typically update their weights linearly with experience i.e., independently of their current strength. This contrasts with biological neurons, which at intermediate strengths are very plastic, but consolidate with Long-Term Potentiation (LTP) once they reach a certain strength. We hypothesize this mechanism might help mitigate catastrophic forgetting. We introduce Sigmoidal Neuronal Adaptive Plasticity (SNAP) an artificial approximation to Long-Term Potentiation for ANNs by having the weights follow a sigmoidal growth behaviour allowing the weights to consolidate and stabilize when they reach sufficiently large or small values. We then compare SNAP to linear weight growth and exponential weight growth and see that SNAP completely prevents the forgetting of previous tasks for Hebbian Learning but not for SGD-base learning.

cs.NE↗
arXiv2024

Articulated Animal AI: An Environment for Animal-like Cognition in a Limbed Agent

This paper presents the Articulated Animal AI Environment for Animal Cognition, an enhanced version of the previous AnimalAI Environment. Key improvements include the addition of agent limbs, enabling more complex behaviors and interactions with the environment that closely resemble real animal movements. The testbench features an integrated curriculum training sequence and evaluation tools, eliminating the need for users to develop their own training programs. Additionally, the tests and training procedures are randomized, which will improve the agent's generalization capabilities. These advancements significantly expand upon the original AnimalAI framework and will be used to evaluate agents on various aspects of animal cognition.

cs.LG↗
arXiv2024

Are Grid Cells Hexagonal for Performance or by Convenience?

This paper investigates whether the hexagonal structure of grid cells provides any performance benefits or if it merely represents a biologically convenient configuration. Utilizing the Vector-HaSH content addressable memory model as a model of the grid cell -- place cell network of the mammalian brain, we compare the performance of square and hexagonal grid cells in tasks of storing and retrieving spatial memories. Our experiments across different path types, path lengths and grid configurations, reveal that hexagonal grid cells perform similarly to square grid cells with respect to spatial representation and memory recall. Our results show comparable accuracy and robustness across different datasets and noise levels on images to recall. These findings suggest that the brain's use of hexagonal grids may be more a matter of biological convenience and ease of implementation rather than because they provide superior performance over square grid cells (which are easier to implement in silico).

q-bio.NC↗
arXiv2022

A Simple Guard for Learned Optimizers

If the trend of learned components eventually outperforming their hand-crafted version continues, learned optimizers will eventually outperform hand-crafted optimizers like SGD or Adam. Even if learned optimizers (L2Os) eventually outpace hand-crafted ones in practice however, they are still not provably convergent and might fail out of distribution. These are the questions addressed here. Currently, learned optimizers frequently outperform generic hand-crafted optimizers (such as gradient descent) at the beginning of learning but they generally plateau after some time while the generic algorithms continue to make progress and often overtake the learned algorithm as Aesop's tortoise which overtakes the hare. L2Os also still have a difficult time generalizing out of distribution. Heaton et al. proposed Safeguarded L2O (GL2O) which can take a learned optimizer and safeguard it with a generic learning algorithm so that by conditionally switching between the two, the resulting algorithm is provably convergent. We propose a new class of Safeguarded L2O, called Loss-Guarded L2O (LGL2O), which is both conceptually simpler and computationally less expensive. The guarding mechanism decides solely based on the expected future loss value of both optimizers. Furthermore, we show theoretical proof of LGL2O's convergence guarantee and empirical results comparing to GL2O and other baselines showing that it combines the best of both L2O and SGD and that in practice converges much better than GL2O.

cs.LG↗
arXiv2020

Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization

Combinatorial optimization has found applications in numerous fields, from aerospace to transportation planning and economics. The goal is to find an optimal solution among a finite set of possibilities. The well-known challenge one faces with combinatorial optimization is the state-space explosion problem: the number of possibilities grows exponentially with the problem size, which makes solving intractable for large problems. In the last years, deep reinforcement learning (DRL) has shown its promise for designing good heuristics dedicated to solve NP-hard combinatorial optimization problems. However, current approaches have two shortcomings: (1) they mainly focus on the standard travelling salesman problem and they cannot be easily extended to other problems, and (2) they only provide an approximate solution with no systematic ways to improve it or to prove optimality. In another context, constraint programming (CP) is a generic tool to solve combinatorial optimization problems. Based on a complete search procedure, it will always find the optimal solution if we allow an execution time large enough. A critical design choice, that makes CP non-trivial to use in practice, is the branching decision, directing how the search space is explored. In this work, we propose a general and hybrid approach, based on DRL and CP, for solving combinatorial optimization problems. The core of our approach is based on a dynamic programming formulation, that acts as a bridge between both techniques. We experimentally show that our solver is efficient to solve two challenging problems: the traveling salesman problem with time windows, and the 4-moments portfolio optimization problem. Results obtained show that the framework introduced outperforms the stand-alone RL and CP solutions, while being competitive with industrial solvers.

cs.AI↗
arXiv2017

Recurrent Ladder Networks

We propose a recurrent extension of the Ladder networks whose structure is motivated by the inference required in hierarchical latent variable models. We demonstrate that the recurrent Ladder is able to handle a wide variety of complex learning tasks that benefit from iterative inference and temporal modeling. The architecture shows close-to-optimal results on temporal modeling of video data, competitive results on music modeling, and improved perceptual grouping based on higher order abstractions, such as stochastic textures and motion cues. We present results for fully supervised, semi-supervised, and unsupervised tasks. The results suggest that the proposed architecture and principles are powerful tools for learning a hierarchy of abstractions, learning iterative inference and handling temporal information.

cs.NE↗
arXiv2012

Emission spectra of self-dual black holes

We calculate the particle spectra of evaporating self-dual black holes that are potential dark matter candidates. We first estimate the relevant mass and temperature range and find that the masses are below the Planck mass, and the temperature of the black holes is small compared to their mass. In this limit, we then derive the number-density of the primary emission particles, and, by studying the wave-equation of a scalar field in the background metric of the black hole, show that we can use the low energy approximation for the greybody factors. We finally arrive at the expression for the spectrum of secondary particle emission from a dark matter halo constituted of self-dual black holes.

gr-qc↗
arXiv2011

Local tuning of Coupling Constants allows for Quantum Fields in Curved Spacetime in the Lab

In this paper we will investigate how one can create emergent curved spacetimes by locally tuning the coupling constants of condensed matter systems. In the continuum limit we thus obtain continuous effective quantum fields living on curved spacetimes. In particular, using Stingnet condensates we can obtain effective electromagnetism. We will show for example how we obtain quantum electromagnetism (U(1)-Yang-Mills) in a black hole (Schwarzschild) spacetime.

gr-qc↗
arXiv2011

Experimenting with Quantum Fields in Curved Spacetime in the Lab

In this paper we will investigate how one can create emergent curved spacetimes by locally tuning the coupling constants of condensed matter systems. In the continuum limit we thus obtain continuous effective quantum fields living on curved spacetimes. In particular, using Stingnet condensates we can obtain effective electromagnetism. We will show for example how we obtain quantum electrodynamics in a blackhole (Schwarzschild) spacetime.

gr-qc↗
arXiv2011

Generalized Uncertainty Principle and Self-dual Black Holes

The Generalized Uncertainty Principle suggests corrections to the Uncertainty Principle as the energy increases towards the Planck value. It provides a natural transition between the expressions for the Compton wavelength below the Planck mass and the black hole event horizon size above this mass. It also suggests corrections to the the event horizon size as the black hole mass falls towards the Planck value, leading to the concept of a Generalized Event Horizon. Extrapolating below the Planck mass suggests the existence of a new class of black holes, whose size is of order the Compton wavelength for their mass. Such sub-Planckian black holes have recently been discovered in the context of loop quantum gravity and it is possible that this applies more generally. This suggests an intriguing connection between black holes, the Uncertainty Principle and quantum gravity.

gr-qc↗
arXiv2010

Lieb-Robinson bounds with dependence on interaction strengths

We propose new Lieb-Robinson bounds (bounds on the speed of propagation of information in quantum systems) with an explicit dependence on the interaction strengths of the Hamiltonian. For systems with more than two interactions it is found that the Lieb-Robinson speed is not always algebraic in the interaction strengths. We consider Hamiltonians with any finite number of bounded operators and also a certain class of unbounded operators. We obtain bounds and propagation speeds for quantum systems on lattices and also general graphs possessing a kind of homogeneity and isotropy. One area for which this formalism could be useful is the study of quantum phase transitions which occur when interactions strengths are varied.

math-ph↗
arXiv2010

A model for non-singular black hole collapse and evaporation

We study the formation of a black hole and its subsequent evaporation in a model employing a minisuperspace approach to loop quantum gravity. In previous work the static solution was obtained and shown to be singularity-free. Here, we examine the more realistic dynamical case by generalizing the static case with help of the Vaidya metric. We track the formation and evolution of trapped surfaces during collapse and evaporation and examine the buildup of quantum gravitationally caused stress-energy preventing the formation of a singularity.

gr-qc↗
arXiv2009

In, Through and Beyond the Planck Scale

In this paper we have recalled the semiclassical metric obtained from a classical analysis of the loop quantum black hole (LQBH). We show that the regular Reissner-Nordström-like metric is self-dual in the sense of T-duality: the form of the metric is invariant under the exchange r -> a0/r where a0 is proportional to the minimum area in LQG. Of particular interest, the symmetry imposes that if an observer at infinity sees a black hole of mass m an observer in the other asymptotic infinity beyond the horizon (near r=0) sees a dual mass proportional to m_P^2/m. We then show that small LQBHs are stable and could be a component of dark matter. Ultra-light LQBHs created shortly after the Big Bang would now have a mass of approximately 10^(-5) m_P and emit radiation with a typical energy of about 10^(13) - 10^(14) eV but they would also emit cosmic rays of much higher energies, albeit few of them. If these small LQBHs form a majority of the dark matter of the Milky Way's Halo, the production rate of ultra-high-energy-cosmic-rays (UHECR) by these ultra light black holes would be compatible with the observed rate of the Auger detector.

hep-th↗
arXiv2009

Self-dual Black Holes in LQG: Theory and Phenomenology

In this paper we have recalled the semiclassical metric obtained from a classical analysis of the loop quantum black hole (LQBH). We show that the regular Reissner-Nordstrom-like metric is self-dual in the sense of T-duality: the form of the metric obtained in Loop quantum Gravity (LQG) is invariant under the exchange "r <-> a0/r" where "a0" is proportional to the minimum area in LQG and "r" is the standard Schwarzschild radial coordinate at asymptotic infinity. Of particular interest, the symmetry imposes that if an observer at "r" close to infinity sees a black hole of mass "m" an observer in the other asymptotic infinity beyond the horizon (at "r" close to "0") sees a dual mass "mp/m" ("mp" is the Planck mass). We then show that small LQBH are stable and could be a component of dark matter. Ultra-light LQBHs created shortly after the Big Bang would now have a mass of approximately "10^(-5) mp" and emit radiation with a typical energy of about 10^(13) - 10^(14) eV but they would also emit cosmic rays of much higher energies, albeit few of them. If these small LQBHs form a majority of the dark matter of the Milky Way's Halo, the production rate of ultra-high-energy-cosmic-rays (UHECR) by these ultra light black holes would be compatible with the observed rate of the Auger detector.

hep-th↗
arXiv2008

Conserved Topological Defects in Non-Embedded Graphs in Quantum Gravity

We follow up on previous work which found that commonly used graph evolution moves lead to conserved quantities that can be expressed in terms of the braiding of the graph in its embedding space. We study non-embedded graphs under three distinct sets of dynamical rules and find non-trivial conserved quantities that can be expressed in terms of topological defects in the dual geometry. For graphs dual to 2-dimensional simplicial complexes we identify all the conserved quantities of the evolution. We also indicate expected results for graphs dual to 3-dimensional simplicial complexes.

gr-qc↗