arXiv subjects

Jean-Pierre Fouque

Publications and source records attributed to Jean-Pierre Fouque.

At least 19 recordsLinked to original sources

arXiv2025

Hashpower allocation in Pay-per-Share blockchain mining pools

Mining blocks in a blockchain using the \textit{Proof-of-Work} consensus protocol involves significant risk, as network participants face continuous operational costs while earning infrequent capital gains upon successfully mining a block. A common risk mitigation strategy is to join a mining pool, which combines the computing resources of multiple miners to provide a more stable income. This article examines a Pay-per-Share (PPS) reward system, where the pool manager can adjust both the share difficulty and the management fee. Using a simplified wealth model for miners, we explore how miners should allocate their computing resources among different mining pools, considering the trade-off between risk transfer to the manager and management fees.

cs.CR↗

arXiv2025

Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces

We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. This algorithm provides solutions to Mean Field Game (MFG) or Mean Field Control (MFC) problems depending on the ratio between two learning rates: one for the value function and the other for the mean field term. In the MFC case, to rigorously identify the limit, we introduce a discretization of the state and action spaces, following the approach used in the finite-space case in [Angiuli et al., 2023b]. The convergence proofs rely on a generalization of the two-timescale framework introduced in [Borkar, 1997]. We further extend our convergence results to Mean Field Control Games, which involve locally cooperative and globally competitive populations. Finally, we present numerical experiments for linear-quadratic problems in one and two dimensions, for which explicit solutions are available.

math.OC↗

arXiv2025

Feynman Formula for Discrete-time Quantum Walks

We explicitly connect (discrete-time) quantum walks on Z with a four-state Markov additive process via a Feynman-type formula (2.5). Using this representation, we derive a relation between the spectral decomposition of the Markov additive process and the limiting density of the homogeneous quantum walk. In addition, we consider a space-time rescaling of quantum walks, which leads to a system of quantum transport PDEs in continuous time and space with a phase interaction term. Our probabilistic representation for this type of PDE offers an efficient Monte Carlo computational technique.

math.PR↗

arXiv2024

Catalan Numbers, Riccati Equations and Convergence

We analyze both finite and infinite systems of Riccati equations derived from stochastic differential games on infinite networks. We discuss a connection to the Catalan numbers and the convergence of the Catalan functions by Fourier transforms.

math.CA↗

arXiv2024

Analysis of Multiscale Reinforcement Q-Learning Algorithms for Mean Field Control Games

Mean Field Control Games (MFCG), introduced in [Angiuli et al., 2022a], represent competitive games between a large number of large collaborative groups of agents in the infinite limit of number and size of groups. In this paper, we prove the convergence of a three-timescale Reinforcement Q-Learning (RL) algorithm to solve MFCG in a model-free approach from the point of view of representative agents. Our analysis uses a Q-table for finite state and action spaces updated at each discrete time-step over an infinite horizon. In [Angiuli et al., 2023], we proved convergence of two-timescale algorithms for MFG and MFC separately highlighting the need to follow multiple population distributions in the MFC case. Here, we integrate this feature for MFCG as well as three rates of update decreasing to zero in the proper ratios. Our technique of proof uses a generalization to three timescales of the two-timescale analysis in [Borkar, 1997]. We give a simple example satisfying the various hypothesis made in the proof of convergence and illustrating the performance of the algorithm.

math.OC↗

arXiv2023

Convergence of Multi-Scale Reinforcement Q-Learning Algorithms for Mean Field Game and Control Problems

We establish the convergence of the unified two-timescale Reinforcement Learning (RL) algorithm presented in a previous work by Angiuli et al. This algorithm provides solutions to Mean Field Game (MFG) or Mean Field Control (MFC) problems depending on the ratio of two learning rates, one for the value function and the other for the mean field term. Our proof of convergence highlights the fact that in the case of MFC several mean field distributions need to be updated and for this reason we present two separate algorithms, one for MFG and one for MFC. We focus on a setting with finite state and action spaces, discrete time and infinite horizon. The proofs of convergence rely on a generalization of the two-timescale approach of Borkar. The accuracy of approximation to the true solutions depends on the smoothing of the policies. We provide a numerical example illustrating the convergence.

math.OC↗

arXiv2023

Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces

We present the development and analysis of a reinforcement learning (RL) algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach pairs the actor-critic (AC) paradigm with a representation of the mean field distribution via a parameterized score function, which can be efficiently updated in an online fashion, and uses Langevin dynamics to obtain samples from the resulting distribution. The AC agent and the score function are updated iteratively to converge, either to the MFG equilibrium or the MFC optimum for a given mean field problem, depending on the choice of learning rates. A straightforward modification of the algorithm allows us to solve mixed mean field control games (MFCGs). The performance of our algorithm is evaluated using linear-quadratic benchmarks in the asymptotic infinite horizon framework.

math.OC↗

arXiv2023

Optimal Trading with Signals and Stochastic Price Impact

Trading frictions are stochastic. They are, moreover, in many instances fast-mean reverting. Here, we study how to optimally trade in a market with stochastic price impact and study approximations to the resulting optimal control problem using singular perturbation methods. We prove, by constructing sub- and super-solutions, that the approximations are accurate to the specified order. Finally, we perform some numerical experiments to illustrate the effect that stochastic trading frictions have on optimal trading.

q-fin.MF↗

arXiv2023

Collective Arbitrage and the Value of Cooperation

We introduce the notions of Collective Arbitrage and of Collective Super-replication in a discrete-time setting where agents are investing in their markets and are allowed to cooperate through exchanges. We accordingly establish versions of the fundamental theorem of asset pricing and of the pricing-hedging duality. A reduction of the price interval of the contingent claims can be obtained by applying the collective super-replication price.

q-fin.MF↗

arXiv2023

Multivariate Systemic Risk Measures and Computation by Deep Learning Algorithms

In this work we propose deep learning-based algorithms for the computation of systemic shortfall risk measures defined via multivariate utility functions. We discuss the key related theoretical aspects, with a particular focus on the fairness properties of primal optima and associated risk allocations. The algorithms we provide allow for learning primal optimizers, optima for the dual representation and corresponding fair risk allocations. We test our algorithms by comparison to a benchmark model, based on a paired exponential utility function, for which we can provide explicit formulas. We also show evidence of convergence in a case for which explicit formulas are not available.

cs.LG↗

arXiv2023

Reinforcement Learning Algorithm for Mixed Mean Field Control Games

We present a new combined \textit{mean field control game} (MFCG) problem which can be interpreted as a competitive game between collaborating groups and its solution as a Nash equilibrium between groups. Players coordinate their strategies within each group. An example is a modification of the classical trader's problem. Groups of traders maximize their wealth. They face cost for their transactions, for their own terminal positions, and for the average holding within their group. The asset price is impacted by the trades of all agents. We propose a three-timescale reinforcement learning algorithm to approximate the solution of such MFCG problems. We test the algorithm on benchmark linear-quadratic specifications for which we provide analytic solutions.

math.OC↗

arXiv2022

Optimal Investment with Correlated Stochastic Volatility Factors

The problem of portfolio allocation in the context of stocks evolving in random environments, that is with volatility and returns depending on random factors, has attracted a lot of attention. The problem of maximizing a power utility at a terminal time with only one random factor can be linearized thanks to a classical distortion transformation. In the present paper, we address the situation with several factors using a perturbation technique around the case where these factors are perfectly correlated reducing the problem to the case with a single factor. Our proposed approximation requires to solve numerically two linear equations in lower dimension instead of a fully non-linear HJB equation. A rigorous accuracy result is derived by constructing sub- and super- solutions so that their difference is at the desired order of accuracy. We illustrate our result with a particular model for which we have explicit formulas for the approximation. In order to keep the notations as explicit as possible, we treat the case with one stock and two factors and we describe an extension to the case with two stocks and two factors.

q-fin.MF↗

arXiv2022

Reinforcement Learning for Intra-and-Inter-Bank Borrowing and Lending Mean Field Control Game

We propose a mean field control game model for the intra-and-inter-bank borrowing and lending problem. This framework allows to study the competitive game arising between groups of collaborative banks. The solution is provided in terms of an asymptotic Nash equilibrium between the groups in the infinite horizon. A three-timescale reinforcement learning algorithm is applied to learn the optimal borrowing and lending strategy in a data driven way when the model is unknown. An empirical numerical analysis shows the importance of the three-timescale, the impact of the exploration strategy when the model is unknown, and the convergence of the algorithm.

math.OC↗

arXiv2022

Deep Learning for Systemic Risk Measures

The aim of this paper is to study a new methodological framework for systemic risk measures by applying deep learning method as a tool to compute the optimal strategy of capital allocations. Under this new framework, systemic risk measures can be interpreted as the minimal amount of cash that secures the aggregated system by allocating capital to the single institutions before aggregating the individual risks. This problem has no explicit solution except in very limited situations. Deep learning is increasingly receiving attention in financial modelings and risk management and we propose our deep learning based algorithms to solve both the primal and dual problems of the risk measures, and thus to learn the fair risk allocations. In particular, our method for the dual problem involves the training philosophy inspired by the well-known Generative Adversarial Networks (GAN) approach and a newly designed direct estimation of Radon-Nikodym derivative. We close the paper with substantial numerical studies of the subject and provide interpretations of the risk allocations associated to the systemic risk measures. In the particular case of exponential preferences, numerical experiments demonstrate excellent performance of the proposed algorithm, when compared with the optimal explicit solution as a benchmark.

q-fin.MF↗

arXiv2022

Systemic Risk Models for Disjoint and Overlapping Groups with Equilibrium Strategies

We analyze the systemic risk for disjoint and overlapping groups (e.g., central clearing counterparties (CCP)) by proposing new models with realistic game features. Specifically, we generalize the systemic risk measure proposed in [F. Biagini, J.-P. Fouque, M. Frittelli, and T. Meyer-Brandis, Finance and Stochastics, 24(2020), 513--564] by allowing individual banks to choose their preferred groups instead of being assigned to certain groups. We introduce the concept of Nash equilibrium for these new models, and analyze the optimal solution under Gaussian distribution of the risk factor. We also provide an explicit solution for the risk allocation of the individual banks, and study the existence and uniqueness of Nash equilibrium both theoretically and numerically. The developed numerical algorithm can simulate scenarios of equilibrium, and we apply it to study the bank-CCP structure with real data and show the validity of the proposed model.

q-fin.MF↗

arXiv2021

Sub- and Super-solution Approach to Accuracy Analysis of Portfolio Optimization Asymptotics in Multiscale Stochastic Factor Market

The problem of portfolio optimization when stochastic factors drive returns and volatilities has been studied in previous works by the authors. In particular, they proposed asymptotic approximations for value functions and optimal strategies in the regime where these factors are running on both slow and fast timescales. However, the rigorous justification of the accuracy of these approximations has been limited to power utilities and a single factor. In this paper, we provide an accurate analysis for cases with general utility functions and two timescale factors by constructing sub- and super-solutions to the fully nonlinear problem so that their difference is at the desired level of accuracy. This approach will be valuable in various related stochastic control problems.

q-fin.MF↗

arXiv2021

Reinforcement Learning for Mean Field Games, with Applications to Economics

Mean field games (MFG) and mean field control problems (MFC) are frameworks to study Nash equilibria or social optima in games with a continuum of agents. These problems can be used to approximate competitive or cooperative games with a large finite number of agents and have found a broad range of applications, in particular in economics. In recent years, the question of learning in MFG and MFC has garnered interest, both as a way to compute solutions and as a way to model how large populations of learners converge to an equilibrium. Of particular interest is the setting where the agents do not know the model, which leads to the development of reinforcement learning (RL) methods. After reviewing the literature on this topic, we present a two timescale approach with RL for MFG and MFC, which relies on a unified Q-learning algorithm. The main novelty of this method is to simultaneously update an action-value function and a distribution but with different rates, in a model-free fashion. Depending on the ratio of the two learning rates, the algorithm learns either the MFG or the MFC solution. To illustrate this method, we apply it to a mean field problem of accumulated consumption in finite horizon with HARA utility function, and to a trader's optimal liquidation problem.

math.OC↗

arXiv2021

Unified Reinforcement Q-Learning for Mean Field Game and Control Problems

We present a Reinforcement Learning (RL) algorithm to solve infinite horizon asymptotic Mean Field Game (MFG) and Mean Field Control (MFC) problems. Our approach can be described as a unified two-timescale Mean Field Q-learning: The \emph{same} algorithm can learn either the MFG or the MFC solution by simply tuning the ratio of two learning parameters. The algorithm is in discrete time and space where the agent not only provides an action to the environment but also a distribution of the state in order to take into account the mean field feature of the problem. Importantly, we assume that the agent can not observe the population's distribution and needs to estimate it in a model-free manner. The asymptotic MFG and MFC problems are also presented in continuous time and space, and compared with classical (non-asymptotic or stationary) MFG and MFC problems. They lead to explicit solutions in the linear-quadratic (LQ) case that are used as benchmarks for the results of our algorithm.

math.OC↗