arXiv subjects

Maxime Gazeau

Publications and source records attributed to Maxime Gazeau.

12 recordsLinked to original sources

arXiv2025

SIMA 2: A Generalist Embodied Agent for Virtual Worlds

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior work (e.g., SIMA 1) limited to simple language commands, SIMA 2 acts as an interactive partner, capable of reasoning about high-level goals, conversing with the user, and handling complex instructions given through language and images. Across a diverse portfolio of games, SIMA 2 substantially closes the gap with human performance and demonstrates robust generalization to previously unseen environments, all while retaining the base model's core reasoning capabilities. Furthermore, we demonstrate a capacity for open-ended self-improvement: by leveraging Gemini to generate tasks and provide rewards, SIMA 2 can autonomously learn new skills from scratch in a new environment. This work validates a path toward creating versatile and continuously learning agents for both virtual and, eventually, physical worlds.

cs.AI↗

arXiv2024

Vision-Language Models as a Source of Rewards

Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting factor for building generalist agents with RL has been the need for a large number of reward functions for achieving different goals. We investigate the feasibility of using off-the-shelf vision-language models, or VLMs, as sources of rewards for reinforcement learning agents. We show how rewards for visual achievement of a variety of language goals can be derived from the CLIP family of models, and used to train RL agents that can achieve a variety of language goals. We showcase this approach in two distinct visual domains and present a scaling trend showing how larger VLMs lead to more accurate rewards for visual goal achievement, which in turn produces more capable RL agents.

cs.LG↗

arXiv2023

A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs

Reasoning on large-scale knowledge graphs has been long dominated by embedding methods. While path-based methods possess the inductive capacity that embeddings lack, their scalability is limited by the exponential number of paths. Here we present A*Net, a scalable path-based method for knowledge graph reasoning. Inspired by the A* algorithm for shortest path problems, our A*Net learns a priority function to select important nodes and edges at each iteration, to reduce time and memory footprint for both training and inference. The ratio of selected nodes and edges can be specified to trade off between performance and efficiency. Experiments on both transductive and inductive knowledge graph reasoning benchmarks show that A*Net achieves competitive performance with existing state-of-the-art path-based methods, while merely visiting 10% nodes and 10% edges at each iteration. On a million-scale dataset ogbl-wikikg2, A*Net not only achieves a new state-of-the-art result, but also converges faster than embedding methods. A*Net is the first path-based method for knowledge graph reasoning at such scale.

cs.AI↗

arXiv2022

In-context Reinforcement Learning with Algorithm Distillation

We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. Unlike sequential policy prediction architectures that distill post-learning or expert sequences, AD is able to improve its policy entirely in-context without updating its network parameters. We demonstrate that AD can reinforcement learn in-context in a variety of environments with sparse rewards, combinatorial task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.

cs.LG↗

arXiv2021

Higher Order Generalization Error for First Order Discretization of Langevin Diffusion

We propose a novel approach to analyze generalization error for discretizations of Langevin diffusion, such as the stochastic gradient Langevin dynamics (SGLD). For an $ε$ tolerance of expected generalization error, it is known that a first order discretization can reach this target if we run $Ω(ε^{-1} \log (ε^{-1}) )$ iterations with $Ω(ε^{-1})$ samples. In this article, we show that with additional smoothness assumptions, even first order methods can achieve arbitrarily runtime complexity. More precisely, for each $N>0$, we provide a sufficient smoothness condition on the loss function such that a first order discretization can reach $ε$ expected generalization error given $Ω( ε^{-1/N} \log (ε^{-1}) )$ iterations with $Ω(ε^{-1})$ samples.

stat.ML↗

arXiv2020

An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise

The choice of batch-size in a stochastic optimization algorithm plays a substantial role for both optimization and generalization. Increasing the batch-size used typically improves optimization but degrades generalization. To address the problem of improving generalization while maintaining optimal convergence in large-batch training, we propose to add covariance noise to the gradients. We demonstrate that the learning performance of our method is more accurately captured by the structure of the covariance matrix of the noise rather than by the variance of gradients. Moreover, over the convex-quadratic, we prove in theory that it can be characterized by the Frobenius norm of the noise matrix. Our empirical studies with standard deep learning model-architectures and datasets shows that our method not only improves generalization performance in large-batch training, but furthermore, does so in a way where the optimization performance remains desirable and the training duration is not elongated.

cs.LG↗

arXiv2019

A general system of differential equations to model first order adaptive algorithms

First order optimization algorithms play a major role in large scale machine learning. A new class of methods, called adaptive algorithms, were recently introduced to adjust iteratively the learning rate for each coordinate. Despite great practical success in deep learning, their behavior and performance on more general loss functions are not well understood. In this paper, we derive a non-autonomous system of differential equations, which is the continuous time limit of adaptive optimization methods. We prove global well-posedness of the system and we investigate the numerical time convergence of its forward Euler approximation. We study, furthermore, the convergence of its trajectories and give conditions under which the differential system, underlying all adaptive algorithms, is suitable for optimization. We discuss convergence to a critical point in the non-convex case and give conditions for the dynamics to avoid saddle points and local maxima. For convex and deterministic loss function, we introduce a suitable Lyapunov functional which allow us to study its rate of convergence. Several other properties of both the continuous and discrete systems are briefly discussed. The differential system studied in the paper is general enough to encompass many other classical algorithms (such as Heavy ball and Nesterov's accelerated method) and allow us to recover several known results for these algorithms.

cs.LG↗

arXiv2018

Scalable Recommender Systems through Recursive Evidence Chains

Recommender systems can be formulated as a matrix completion problem, predicting ratings from user and item parameter vectors. Optimizing these parameters by subsampling data becomes difficult as the number of users and items grows. We develop a novel approach to generate all latent variables on demand from the ratings matrix itself and a fixed pool of parameters. We estimate missing ratings using chains of evidence that link them to a small set of prototypical users and items. Our model automatically addresses the cold-start and online learning problems by combining information across both users and items. We investigate the scaling behavior of this model, and demonstrate competitive results with respect to current matrix factorization techniques in terms of accuracy and convergence speed.

cs.IR↗

arXiv2018

Bloch theory and spectral gaps for linearized water waves

The system of equations for water waves, when linearized about equilibrium of a fluid body with a varying bottom boundary, is described by a spectral problem for the Dirichlet -- Neumann operator of the unperturbed free surface. This spectral problem is fundamental in questions of stability, as well as to the perturbation theory of evolution of the free surface in such settings. In addition, the Dirichlet -- Neumann operator is self-adjoint when given an appropriate definition and domain, and it is a novel but very natural spectral problem for a nonlocal operator. In the case in which the bottom boundary varies periodically, $\{y = -h + b(x)\}$ where $b(x+γ) = b(x)$, $γ\in Γ$ a lattice, this spectral problem admits a Bloch decomposition in terms of spectral band functions and their associated band-parametrized eigenfunctions. In this article we describe this analytic construction in the case of a spatially periodic bottom variation from constant depth in two space dimensional water waves problem, giving a construction of the Bloch eigenfunctions and eigenvalues as a function of the band parameters and a description of the Dirichlet -- Neumann operator in terms of the bathymetry $b(x)$. One of the consequences of this description is that the spectrum consists of a series of bands separated by spectral gaps which are zones of forbidden energies. For a given generic periodic bottom profile $b(x)=\varepsilon β(x)$, every gap opens for a sufficiently small value of the perturbation parameter $\varepsilon$.

math.AP↗

arXiv2017

Implicit Manifold Learning on Generative Adversarial Networks

This paper raises an implicit manifold learning perspective in Generative Adversarial Networks (GANs), by studying how the support of the learned distribution, modelled as a submanifold $\mathcal{M}_θ$, perfectly match with $\mathcal{M}_{r}$, the support of the real data distribution. We show that optimizing Jensen-Shannon divergence forces $\mathcal{M}_θ$ to perfectly match with $\mathcal{M}_{r}$, while optimizing Wasserstein distance does not. On the other hand, by comparing the gradients of the Jensen-Shannon divergence and the Wasserstein distances ($W_1$ and $W_2^2$) in their primal forms, we conjecture that Wasserstein $W_2^2$ may enjoy desirable properties such as reduced mode collapse. It is therefore interesting to design new distances that inherit the best from both distances.

stat.ML↗

arXiv2014

Analysis and simulation of rare events for SPDE

In this work, we consider the numerical estimation of the probability for a stochastic process to hit a set B before reaching another set A. This event is assumed to be rare. We consider reactive trajectories of the stochastic Allen-Cahn partial differential evolution equation (with double well potential) in dimension 1. Reactive trajectories are defined as the probability distribution of the trajectories of a stochastic process, conditioned by the event of hitting B before A. We investigate the use of the so-called Adaptive Multilevel Splitting algorithm in order to estimate the rare event and simulate reactive trajectories. This algorithm uses a \emph{reaction coordinate} (a real valued function of state space defining level sets), and is based on (i) the selection, among several replicas of the system having hit A before B, of those with maximal reaction coordinate; (ii) iteration of the latter step. We choose for the reaction coordinate the average magnetization, and for B the minimum of the well opposite to the initial condition. We discuss the context, prove that the algorithm has a sense in the usual functional setting, and numerically test the method (estimation of rare event, and transition state sampling).

math.AP↗

arXiv2013

Strong order of convergence of a semidiscrete scheme for the stochastic Manakov equation

It is well accepted by physicists that the Manakov PMD equation is a good model to describe the evolution of nonlinear electric fields in optical fibers with randomly varying birefringence. In the regime of the diffusion approximation theory, an effective asymptotic dynamics has recently been obtained to describe this evolution. This equation is called the stochastic Manakov equation. In this article, we propose a semidiscrete version of a Crank Nicolson scheme for this limit equation and we analyze the strong error. Allowing sufficient regularity of the initial data, we prove that the numerical scheme has strong order 1/2.

math.NA↗