arXiv subjects

Mathieu Laurière

Publications and source records attributed to Mathieu Laurière.

83 records · Page 5Linked to original sources

arXiv2020

On the Convergence of Model Free Learning in Mean Field Games

Learning by experience in Multi-Agent Systems (MAS) is a difficult and exciting task, due to the lack of stationarity of the environment, whose dynamics evolves as the population learns. In order to design scalable algorithms for systems with a large population of interacting agents (e.g. swarms), this paper focuses on Mean Field MAS, where the number of agents is asymptotically infinite. Recently, a very active burgeoning field studies the effects of diverse reinforcement learning algorithms for agents with no prior information on a stationary Mean Field Game (MFG) and learn their policy through repeated experience. We adopt a high perspective on this problem and analyze in full generality the convergence of a fictitious iterative scheme using any single agent learning algorithm at each step. We quantify the quality of the computed approximate Nash equilibrium, in terms of the accumulated errors arising at each learning iteration step. Notably, we show for the first time convergence of model free learning algorithms towards non-stationary MFG equilibria, relying only on classical assumptions on the MFG dynamics. We illustrate our theoretical results with a numerical experiment in a continuous action-space environment, where the approximate best response of the iterative fictitious play scheme is computed with a deep RL algorithm.

math.OC↗

arXiv2020

Large Banking Systems with Default and Recovery: A Mean Field Game Model

We consider a mean-field model for large banking systems, which takes into account default and recovery of the institutions. Building on models used for groups of interacting neurons, we first study a McKean-Vlasov dynamics and its evolutionary Fokker-Planck equation in which the mean-field interactions occur through a mean-reverting term and through a hitting time corresponding to a default level. The latter feature reflects the impact of a financial institution's default on the global distribution of reserves in the banking system. The systemic risk problem of financial institutions is understood as a blow-up phenomenon of the Fokker-Planck equation. Then, we incorporate in the model an optimization component by letting the institutions control part of their dynamics in order to minimize their expected risk. Phrasing this optimization problem as a mean-field game, we provide an explicit solution in a special case and, in the general case, we report numerical experiments based on a finite difference scheme.

math.OC↗

arXiv2019

Optimal control of conditioned processes with feedback controls

We consider a class of closed loop stochastic optimal control problems in finite time horizon, in which the cost is an expectation conditional on the event that the process has not exited a given bounded domain. An important difficulty is that the probability of the event that conditionates the strategy decays as time grows. The optimality conditions consist of a system of partial differential equations, including a Hamilton-Jacobi-Bellman equation (backward w.r.t. time) and a (forward w.r.t. time) Fokker-Planck equation for the law of the conditioned process. The two equations are supplemented with Dirichlet conditions. Next, we discuss the asymptotic behavior as the time horizon tends to $+\infty$. This leads to a new kind of optimal control problem driven by an eigenvalue problem related to a continuity equation with Dirichlet conditions on the boundary. We prove existence for the latter. We also propose numerical methods and supplement the various theoretical aspects with numerical simulations.

math.OC↗

arXiv2019

Backward propagation of chaos

This paper develops a theory of propagation of chaos for a system of weakly interacting particles whose terminal configuration is fixed as opposed to the initial configuration as customary. Such systems are modeled by backward stochastic differential equations. Under standard assumptions on the coefficients of the equations, we prove propagation of chaos results and quantitative estimates on the rate of convergence in Wasserstein distance of the empirical measure of the interacting system to the law of a McKean-Vlasov type equation. These results are accompanied by non-asymptotic concentration inequalities. As an application, we derive rate of convergence results for solutions of second order semilinear partial differential equations to the solution of a partial differential written on an infinite dimensional space.

math.PR↗

arXiv2018

On the implementation of a primal-dual algorithm for second order time-dependent mean field games with local couplings

We study a numerical approximation of a time-dependent Mean Field Game (MFG) system with local couplings. The discretization we consider stems from a variational approach described in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] for the stationary problem and leads to the finite difference scheme introduced by Achdou and Capuzzo-Dolcetta in [SIAM J. Numer. Anal., 48(3):1136-1162, 2010]. In order to solve the finite dimensional variational problems, in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] the authors implement the primal-dual algorithm introduced by Chambolle and Pock in [J. Math. Imaging Vision, 40(1):120-145, 2011], whose core consists in iteratively solving linear systems and applying a proximity operator. We apply that method to time-dependent MFG and, for large viscosity parameters, we improve the linear system solution by replacing the direct approach used in [Briceno-Arias, Kalise, and Silva, SIAM J. Control Optim., 2017] by suitable preconditioned iterative algorithms.

math.OC↗

arXiv2018

Mean Field Control and Mean Field Game Models with Several Populations

In this paper, we investigate the interaction of two populations with a large number of indistinguishable agents. The problem consists in two levels: the interaction between agents of a same population, and the interaction between the two populations. In the spirit of mean field type control (MFC) problems and mean field games (MFG), each population is approximated by a continuum of infinitesimal agents. We define four different problems in a general context and interpret them in the framework of MFC or MFG. By calculus of variations, we derive formally in each case the adjoint equations for the necessary conditions of optimality. Importantly, we find that in the case of a competition between two coalitions, one needs to rely on a system of Master equations in order to describe the equilibrium. Examples are provided, in particular linear-quadratic models for which we obtain systems of ODEs that can be related to Riccati equations.

math.OC↗

arXiv2018

A Class of Finite-Dimensional Numerically Solvable McKean-Vlasov Control Problems

We address a class of McKean-Vlasov (MKV) control problems with common noise, called polynomial conditional MKV, and extending the known class of linear quadratic stochastic MKV control problems. We show how this polynomial class can be reduced by suitable Markov embedding to finite-dimensional stochastic control problems, and provide a discussion and comparison of three probabilistic numerical methods for solving the reduced control problem: quantization, regression by control randomization, and regress later methods. Our numerical results are illustrated on various examples from portfolio selection and liquidation under drift uncertainty, and a model of interbank systemic risk with partial observation.

math.OC↗

arXiv2018

Robust Bell inequalities from communication complexity

The question of how large Bell inequality violations can be, for quantum distributions, has been the object of much work in the past several years. We say that a Bell inequality is normalized if its absolute value does not exceed 1 for any classical (i.e. local) distribution. Upper and (almost) tight lower bounds have been given for the quantum violation of these Bell inequalities in terms of number of outputs of the distribution, number of inputs, and the dimension of the shared quantum states. In this work, we revisit normalized Bell inequalities together with another family: inefficiency-resistant Bell inequalities. To be inefficiency-resistant, the Bell value must not exceed 1 for any local distribution, including those that can abort. This makes the Bell inequality resistant to the detection loophole, while a normalized Bell inequality is resistant to general local noise. Both these families of Bell inequalities are closely related to communication complexity lower bounds. We show how to derive large violations from any gap between classical and quantum communication complexity, provided the lower bound on classical communication is proven using these lower bound techniques. This leads to inefficiency-resistant violations that can be exponential in the size of the inputs. Finally, we study resistance to noise and inefficiency for these Bell inequalities.

quant-ph↗

arXiv2017

The information cost of quantum memoryless protocols

We consider memoryless quantum communication protocols, where the two parties do not possess any memory besides their classical input and they take turns performing unitary operations on a pure quantum state that they exchange between them. Most known quantum protocols are of this type and recently a deep connection between memoryless protocols and Bell inequality violations has been explored recently by Buhrman et al. We study the information cost of memoryless quantum protocols by looking at a canonical problem: bounded-round quantum communication protocols for the one-bit AND function. We prove directly a tight lower bound of $Θ(\log(k) / k)$ for the information cost of AND for $k$-round memoryless quantum protocols and for the input distribution needed for the Disjointness function. It is not clear if memoryless protocols allow for a reduction between AND and Disjointness, due to the absence of private workspaces. We enhance the model by allowing the players to keep in their private classical workspace apart from their input also some classical private coins. Surprisingly, we show that every quantum protocol can be transformed into an equivalent quantum protocol with private coins that is perfectly private, i.e. the players only learn the value of the function and nothing more. Last, we consider the model where the players are allowed to use one-shot coins, i.e. private coins that can be used only once during the protocol. While in the classical case, private coins and one-shot coins are equivalent, in the quantum case, we prove that they are not. More precisely, we show that every quantum memoryless protocol with one-bit inputs that uses one-shot coins can be transformed into a memoryless quantum protocol without private coins and without increasing too much its information cost. Hence, while private coins always allow for private quantum protocols, one-shot coins do not.

quant-ph↗

arXiv2016

Extended Learning Graphs for Triangle Finding

We present new quantum algorithms for Triangle Finding improving its best previously known quantum query complexities for both dense and spare instances.For dense graphs on $n$ vertices, we get a query complexity of $O(n^{5/4})$ without any of the extra logarithmic factors present in the previous algorithm of Le Gall [FOCS'14]. For sparse graphs with $m\geq n^{5/4}$ edges, we get a query complexity of $O(n^{11/12}m^{1/6}\sqrt{\log n})$, which is better than the one obtained by Le Gall and Nakajima [ISAAC'15] when $m \geq n^{3/2}$. We also obtain an algorithm with query complexity ${O}(n^{5/6}(m\log n)^{1/6}+d_2\sqrt{n})$ where $d_2$ is the variance of the degree distribution. Our algorithms are designed and analyzed in a new model of learning graphs that we call extended learning graphs. In addition, we present a framework in order to easily combine and analyze them. As a consequence we get much simpler algorithms and analyses than previous algorithms of Le Gall {\it et al} based on the MNRS quantum walk framework [SICOMP'11].

quant-ph↗

arXiv2014

Privacy in Quantum Communication Complexity

In two-party quantum communication complexity, Alice and Bob receive some classical inputs and wish to compute some function that depends on both these inputs, while minimizing the communication. This model has found numerous applications in many areas of computer science. One question that has received a lot of attention recently is whether it is possible to perform such protocols in a private way. We show that defining privacy for quantum protocols is not so straightforward and it depends on whether we assume that the registers where Alice and Bob receive their classical inputs are in fact classical registers (and hence unentangled with the rest of the protocol) or quantum registers (and hence can be entangled with the rest of the protocol or the environment). We provide new quantum protocols for the Inner Product function and for Private Information Retrieval, and show that the privacy assuming classical input registers can be exponentially smaller than the privacy assuming quantum input registers. We also argue that the right notion of privacy of a communication protocol is the one assuming classical input registers, since otherwise the players can deviate considerably from the protocol.

quant-ph↗