ABIDES-MARL: A Multi-Agent Reinforcement Learning Environment for Optimal Execution with Endogenous Liquidity

Abstract

Classical optimal execution models treat market impact as a pre-specified, exogenous process. However, when market makers adapt strategically, this assumption becomes a structural misspecification: execution dynamics depend on the policies and actions of other agents. The problem therefore ceases to be a single-agent control problem and instead becomes a finite-horizon stochastic game, in which liquidity emerges endogenously from the interactions among heterogeneous market players. We hence introduce ABIDES-MARL, a multi-agent reinforcement learning framework for studying optimal execution under endogenous liquidity in a realistic limit order book setting. The framework extends ABIDES-Gym to support multiple learning agents with synchronized decision periods that preserve proper information filtration and key market microstructure features. We validate the framework in an extended Kyle setting, where multiple learning agents recovers gradual price discovery. Numerical results then show that execution strategies learned under endogenous, adaptive liquidity differ fundamentally from those implied by fixed exogenous price-impact benchmarks. In particular, strategies optimized under exogenous impact perform poorly once market makers adapt strategically: when information is not balanced across traders and market makers, market makers exploit predictable execution and the market dynamics may degenerate. The framework therefore provides a reproducible foundation for studying strategic adaptation in realistic markets and contributes to the development of economically interpretable agentic AI systems in finance.