Searcharxiv⌕ Search

arXiv subjects

Mathieu Laurière

Publications and source records attributed to Mathieu Laurière.

At least 19 recordsLinked to original sources
arXiv2026

Mean Field Reinforcement Learning: Theory and Algorithms for Mean Field Control

This monograph provides an introduction to mean field reinforcement learning through the lens of Markov decision processes arising from large-population stochastic control with mean field interactions and common noise. Starting from the connection between multi-agent reinforcement learning and mean field control, it develops the probabilistic, mathematical, and control-theoretic framework needed to formulate representative-agent learning problems, analyze their relationship with finite-population systems, and study both general and linear-quadratic models. The presentation includes dynamic programming principles, propagation-of-chaos limits, and theoretical analyses of tabular Q-learning and policy-gradient methods. It also discusses numerical implementations, including tabular schemes and deep reinforcement learning methods such as deep deterministic policy gradient. The goal is to give readers a coherent bridge between mean field control theory and reinforcement learning methodology, emphasizing the mathematical structure of the problems and the design of tractable learning approaches for large stochastic populations.

math.OC↗
arXiv2026

Optimization Frameworks & Learning Algorithms for Discrete-Time Mean Field Games

This monograph introduces discrete-time mean field games with finite state and action spaces, emphasizing optimization formulations and learning algorithms. Starting from Markov decision processes and finite-player stochastic games, it develops the notion of mean field Nash equilibrium, establishes existence and uniqueness results under suitable assumptions, and explains how mean field equilibria yield approximate equilibria in large finite-player games. Building on the linear programming formulation of Markov decision processes, occupation measures and primal-dual formulations connect equilibrium computation with mathematical optimization. The monograph presents fixed-point iterations, fictitious play, online mirror descent, and optimization-based methods, discussing their convergence properties and limitations. Numerical examples illustrate crowd motion, traffic routing, epidemic management, and the computation of multiple equilibria. It combines examples, theoretical results, and implementable algorithms, with bibliographic notes providing connections to the broader literature.

math.OC↗
arXiv2026

Discrete-Time Mean Field Type Games: Probabilistic Setup

We introduce a general probabilistic framework for discrete-time, infinite-horizon discounted Mean Field Type Games (MFTGs) with both global common noise and team-specific common noises. In our model, agents are allowed to use randomized actions, both at the individual level and at the team level. We formalize the concept of Mean Field Markov Games (MFMGs) and establish a connection between closed-loop policies in MFTGs and Markov policies in MFMGs through different layers of randomization. By leveraging recent results on infinite-horizon discounted games with infinite compact state-action spaces, we prove the existence of a stationary Nash-equilibrium profile of closed-loop policies for the original MFTG when the state spaces and the action spaces are compact Polish spaces. We also present an example satisfying our assumptions.

math.OC↗
arXiv2026

Bayesian Learning of Parametric Mean Field Games with Common Noise

We study the inverse problem of recovering unknown parameters governing the dynamics and rewards of Mean Field Games (MFGs) from partial observations of population trajectories, made stochastic by common noise. The observer seeks to recover a consistent parameter $θ$ via Bayesian inference, facing the challenge that evaluating the likelihood requires solving a new Nash equilibrium (NE) for each candidate $θ$. Due to common noise, the mean-field evolution is stochastic and equilibrium policies are population dependent, which makes solving for each parameter computationally expensive. To overcome this, we introduce the Bayesian-Extended Mean Field Game (BE-MFG), whose single NE simultaneously encodes equilibrium policies for all parameters, enabling efficient learning of the conditional density of mean-field trajectories. We formalize parameter consistency and identifiability in MFGs with common noise and investigate conditions under which consistent parameter recovery is guaranteed. On three synthetic environments of various complexity, we recover the ground-truth parameter accurately from a small number of partial, stochastic observations, outperforming an adaptation of inverse reinforcement learning for MFGs. A fourth experiment, on public electricity market data, suggests that the approach can also be applied to real-world data.

cs.GT↗
arXiv2026

Inverse Problems for Costs and Controls in LQG MFGs via Mean Field Trajectories

This paper investigates inverse problems for Linear-Quadratic-Gaussian (LQG) Mean Field Games (MFGs) based entirely on the observation of mean-covariance trajectories. We address three sequential challenges: identifying the optimal control for observed initializations, determining the control for arbitrary initializations, and recovering consistent cost parameters. After establishing the existence and uniqueness of the forward Nash equilibrium under mild hypotheses, we analyze the injectivity of the parameter-to-trajectory mapping, demonstrating that it is inherently non-injective and providing sufficient conditions for parameter equivalence. We prove that while the optimal control is locally identifiable for observed initializations under minimal assumptions, global identifiability requires a deeper structural recovery of the game's costs. To bridge this gap, we propose a constructive semidefinite programming method to infer cost parameters that are strictly consistent with the observed population dynamics. Numerical experiments illustrate this method.

math.OC↗
arXiv2026

Extended Mean Field Control Games with Moment Interactions: General Framework and Linear-Quadratic Model

Mean field control games (MFCGs) provide a framework for studying non-cooperative games involving groups of cooperative players in the limit as both the number of players and the number of groups go to infinity.This class of games includes mean field games (MFGs) and mean field control (MFC) problems as special cases, corresponding to purely non-cooperative and purely cooperative settings, respectively. We incorporate interactions through the mean of actions, hence the terminology of extended MFCGs. We first show that the equilibrium control can be described using a coupled system of partial differential equations consisting of a Hamilton-Jacobi-Bellman equation and a Fokker-Planck equation. After introducing the general framework, we focus on a linear-quadratic (LQ) structure. We show that the equilibrium can be reduced to a system of ODEs, generalizing those obtained for MFGs and MFC problems. We then provide two numerical examples illustrating specific features of MFCGs.

math.OC↗
arXiv2026

Extended Graphon Mean-Field Games in Discrete Time

In this paper, we study games involving a continuum of heterogeneous players in the discrete-time setting with finite state spaces and continuous action spaces. We introduce a new model that incorporates joint state-action interactions within the graphon-weighted aggregate, described by a coupled forward-backward system. We establish the existence of graphon mean-field equilibria and characterize them through this forward-backward system. Additionally, we provide uniqueness results under monotonicity and contraction conditions. To illustrate the practical relevance of our framework, we solve an example of portfolio liquidation with price impact. We provide numerical results for three different graphons and two different initial distributions, illustrating the impact of the network's structure and the distribution heterogeneity on the distribution and the policy.

math.OC↗
arXiv2026

Towards Scaling Reinforcement Learning to Massive Populations: Learning Mean-Field Representations

Modern multi-agent systems are increasingly deployed at scale over large populations of agents in settings such as ad-auctions, traffic routing, and recommendation systems. The dominant approach in such settings is to optimize each agent's policy independently, treating the other agents as part of a fixed single-agent environment rather than modeling the population dynamics. In many large-population systems, the dynamics depend on an aggregate summary of the population rather than the identity of any individual. Mean-field RL exploits such structure, providing a principled framework that models each agent's environment as an explicit function of the population distribution. However, in large state-action spaces or high-dimensional control problems, modeling the population distribution is itself intractable. How can we design a scalable framework for high-dimensional control problems with large populations? This work explores this question from the perspective of representation learning. We introduce a mean-field RL framework in which the rewards and transition dynamics depend on the population only through an unknown low-dimensional aggregate statistic. We then study this framework in the offline setting and design a provable approach that learns a near-optimal policy by learning a low-dimensional representation. Motivated by real-life supply-chain optimization problems, we design a one-step routing game to test the hypothesis that learning a low-dimensional population representation improves reward prediction and Nash gap estimation relative to baselines that don't exploit this structure. We show that under a fixed neural-network parameter count and optimization budget, learning a low-dimensional population representation improves reward prediction and the equilibrium quality of the resulting policies.

cs.MA↗
arXiv2026

Self-fictitious-play for Potential Monotone Ergodic Mean-field Games

We investigate long-time learning in ergodic, potential, monotone mean-field games (MFGs) via a self-fictitious-play (SFP) dynamics coupling an optimally controlled diffusion with a slowly evolving belief. At each time, the state follows the optimal feedback associated with the current belief, while the belief is updated using the player's own empirical occupation measure rather than the population distribution. For ergodic monotone potential MFGs on the torus, we prove that the SFP dynamics is contractive and admits a unique invariant law. Moreover, we show that this invariant law is quantitatively close to the MFG Nash equilibrium, with an error of order equal to the square root of the belief-update rate. The proof combines uniform-in-time regularity estimates for the ergodic Hamilton-Jacobi-Bellman equation with an energy argument based on the Lasry-Lions divergence. The linear-quadratic example shows that this rate is sharp, and the numerical experiments illustrate the predicted scaling.

math.OC↗
arXiv2026

Nonlocal Mean Field Schrödinger Bridge with Learned Interactions

The Schrödinger Bridge Problem connects an initial distribution to a terminal one along a minimum-energy stochastic process. Its mean-field extension, the Mean-Field Schrödinger Bridge, governs interacting populations whose dynamics and costs depend on the collective distribution. When these interactions are nonlocal, their direct evaluation scales quadratically with the population size, making large ensembles intractable within FBSDE-based solvers. We replace these terms with neural surrogates in state and time, trained on empirical interaction values along sampled trajectories and embedded in a four-stage alternating scheme that updates the forward and backward potentials and the surrogates in turn, while preserving forward--backward consistency and the prescribed endpoint marginals. We derive Grönwall-type stability bounds quantifying how surrogate errors propagate to the generated trajectories under a small-gain condition. On crowd-navigation and high-dimensional opinion-dynamics benchmarks, the surrogates reproduce the trajectories obtained with exact evaluation at reduced training cost. The advantage is most significant when the interaction is a nonlinear functional of the measure, such as the normalized bounded-confidence drift, for which random-batch subsampling is biased and unstable whereas the learned surrogate remains accurate.

math.OC↗
arXiv2026

Neural Parameter Calibration for Finite-State Mean Field Games

Mean field games efficiently approximate a very large population of strategic agents. While these games can aid the understanding of complex systems, their deployment in real-world settings is challenged by the specification of their parameters: mean field games (MFGs) often involve hidden preferences, constraints, and interactions that can rarely be theoretically derived or directly observed. To address this gap, we present a neural network-based framework for learning parametric, finite-state MFGs from observed population dynamics. To do so, we formulate the parameter calibration as an inverse problem and use implicit differentiation to backpropagate through the games' equilibrium. The resulting approach is fully differentiable and enables us to estimate flexible trajectory-wise parameter paths, including state- and time-dependent specifications without requiring observations of the individual agents' actions or rewards. We provide a proof for the exactness of the gradient computation in a discrete-time formulation. We validate our framework through numerical experiments across four systems of increasing complexity, ranging from synthetic linear-quadratic benchmarks to real-world urban mobility datasets.

cs.GT↗
arXiv2026

Robust $Q$-learning for mean-field control under Wasserstein uncertainty in common noise

In this article, we present a robust $Q$-learning algorithm for discrete-time mean-field control problems under Wasserstein uncertainty in the common noise law. The algorithm combines a quantization-and-projection scheme with a Wasserstein dual reformulation on the common-noise space. We establish its convergence together with finite-time iteration bounds for both synchronous and asynchronous learning schemes. Numerical experiments on systemic risk and epidemic models compare the asynchronous implementation with an idealized Bellman iteration, illustrate the robustness-performance tradeoff under common-noise misspecification, and report the observed convergence behavior of the asynchronous $Q$-learning algorithm.

math.OC↗
arXiv2026

Population-Aware Imitation Learning in Mean-field Games with Common Noise

Mean Field Games (MFGs) provide a powerful framework for modeling the collective behavior of large populations of interacting agents. In this paper, we address the problem of Imitation Learning (IL) in MFGs subject to common noise, where the population distribution evolves stochastically. This stochasticity compels agents to adopt population-aware policies to respond to aggregate shocks. We formulate two distinct learning objectives: recovering a Nash equilibrium and maximizing performance against an expert population. We investigate two imitation proxies: Behavioral Cloning (BC) and Adversarial (ADV) divergence. We then establish finite-sample error bounds showing that minimizing these proxies effectively controls both the policy's exploitability and its performance gap relative to the expert. Furthermore, we propose a numerical framework using generalized Fictitious Play and Deep Learning to compute expert population-aware policies. Through experiments on three environments we demonstrate that standard population-unaware policies fail to capture the equilibrium dynamics. Our results highlight that learning population-aware policies is crucial to avoid being misled by the randomness inherent in common noise.

cs.LG↗
arXiv2026

Particle Methods with Deep Learning for Stochastic Control under Partial Observation

Numerical computation of stochastic control problems under partial observation is challenging because the dynamic programming formulation is naturally posed on the conditional distribution of the hidden state. We propose particle-based methods that replace this infinite-dimensional filtering state by a finite-dimensional weighted particle system, building on recent limit theory for mean-field control with common-noise-adapted controls. We prove, under suitable assumptions, convergence of the fully discretized particle approximation to the original continuous-time partially observed control problem. The particle reformulation is high-dimensional but permutation-invariant, a structure that can be exploited by symmetric neural network architectures. We develop two deep learning algorithms: a direct optimization method for feedback controls and a Deep BSDE method for particle problems admitting a backward stochastic differential equation representation. We also extend the computational framework to partially observed mean-field control problems, which have been studied theoretically but remain less developed numerically. Numerical experiments on a linear--quadratic benchmark, a nonlinear partially observed mean-field control problem, and two financial applications, portfolio liquidation and asset allocation, demonstrate the accuracy and practical utility of the approach.

math.OC↗
arXiv2026

MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs

Large language models (LLMs) are increasingly deployed as interactive agents, yet their capacity for social and strategic reasoning over extended interaction remains poorly understood. Existing evaluations rely on static vignettes or single-game benchmarks that cannot capture the sustained, multi-faceted reasoning that real-world multi-agent settings demand. We introduce Mindgames, a multi-game arena and evaluation platform for LLM agents that operationalizes complementary reasoning demands relevant to ``theory of mind'': belief attribution under hidden information, opponent modeling through repeated strategic interaction, cooperative inference under knowledge asymmetries, and sustained deception in social deduction. Built on TextArena, Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments. We instantiate Mindgames through a 2025 competition cycle hosted at a major AI conference, which assessed 944 submitted agents from 76 teams across four games: Colonel Blotto, Iterated Prisoner's Dilemma, Codenames, and Secret Mafia. Our analysis surfaces both agent-level and evaluation-level limitations: brittle rule adherence remains a major bottleneck, top-performing systems repeatedly rely on explicit structural scaffolding, and leaderboard validity differs sharply across environments. In particular, failure-heavy environments can reward robustness to opponent errors as much as strategic ability, with Secret Mafia exhibiting a pronounced error-survival confound in this cycle. We release a dataset of 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol that scores new agents against a frozen reference pool of top-ranked, low-error Stage~II submissions under the same error-attribution lens used in this analysis.

cs.AI↗
arXiv2026

Scalable method for mean field control with kernel interactions via random Fourier features

We develop a scalable algorithm for mean field control problems with kernel interactions by combining particle system simulations with random Fourier feature approximations. The method replaces the quadratic-cost kernel evaluations by linear-time estimates, enabling efficient stochastic gradient descent for training feedback controls in large populations. We provide theoretical complexity bounds and demonstrate through crowd motion and flocking examples that the approach preserves control performance while substantially reducing computational cost. The results indicate that random feature approximations offer an effective and practical tool for high dimensional and large scale mean field control.

math.OC↗
arXiv2026

BAGEL: Benchmarking Animal Knowledge Expertise in Language Models

Large language models have shown strong performance on broad-domain knowledge and reasoning benchmarks, but it remains unclear how well language models handle specialized animal-related knowledge under a unified closed-book evaluation protocol. We introduce BAGEL, a benchmark for evaluating animal knowledge expertise in language models. BAGEL is constructed from diverse scientific and reference sources, including bioRxiv, Global Biotic Interactions, Xeno-canto, and Wikipedia, using a combination of curated examples and automatically generated closed-book question-answer pairs. The benchmark covers multiple aspects of animal knowledge, including taxonomy, morphology, habitat, behavior, vocalization, geographic distribution, and species interactions. By focusing on closed-book evaluation, BAGEL measures animal-related knowledge of models without external retrieval at inference time. BAGEL further supports fine-grained analysis across source domains, taxonomic groups, and knowledge categories, enabling a more precise characterization of model strengths and systematic failure modes. Our benchmark provides a new testbed for studying domain-specific knowledge generalization in language models and for improving their reliability in biodiversity-related applications.

cs.CL↗
arXiv2026

Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formula

We develop dual approaches for continuous-time stochastic control problems, enabling the computation of robust dual bounds in high-dimensional state and control spaces. Building on the dual formulation proposed in [L. C. G. Rogers, SIAM Journal on Control and Optimization, 46 (2007), pp. 1116--1132], we first formulate the inner optimization problem as a stochastic partial differential equation (SPDE); the expectation of its solution yields the dual bound. Curse-of-dimensionality-free methods are proposed based on the Pontryagin maximum principle and the generalized Hopf formula. In the process, we prove the generalized Hopf formula, first introduced as a conjecture in [Y. T. Chow, J. Darbon, S. Osher, and W. Yin, Journal of Computational Physics 387 (2019), pp. 376--409], under mild conditions. Numerical experiments demonstrate that our dual approaches effectively complement primal methods, including the deep BSDE method for solving high-dimensional PDEs and the deep actor-critic method in reinforcement learning.

math.OC↗