SearcharxivSearch

arXiv subjects

Joe Jackson

Publications and source records attributed to Joe Jackson.

At least 19 recordsLinked to original sources

The Mean-Field Limit of Online Stochastic Vector Balancing

We study an online vector balancing problem, in which $n$ independent Gaussian random vectors $\boldsymbol{\zeta}(1),\dots,\boldsymbol{\zeta}(n) \sim \mathcal{N}(0, I_n)$, each of dimension $n$, arrive one at a time. The goal is to choose signs $\varepsilon(1),\dots,\varepsilon(n) \in \{\pm 1\}$ with $\varepsilon(k)$ depending only on $\boldsymbol{\zeta}(1),\dots,\boldsymbol{\zeta}(k)$, so as to minimize the expected $\ell^{\infty}$ norm of the signed sum $\frac{1}{\sqrt{n}}\sum_{k = 1}^n \varepsilon(k) \boldsymbol{\zeta}(k)$. Prior work showed that the optimal value $V^n$ is $O(1)$, at least for Rademacher $\boldsymbol{\zeta}(k)$'s, by constructing specific algorithms. Our main contribution is to determine the exact limit $V^{\infty} = \lim_{n\to\infty} V^n$ as the value of a nonstandard stochastic control problem of mean-field type: find the narrowest terminal interval into which a Brownian motion can be adaptively steered under a uniform-in-time $L^2$ constraint on the drift. The proof of the lower bound $V^{\infty} \leq \liminf_{n \to \infty} V^n$ uses probabilistic compactness arguments, and is very flexible. In fact, we show that the lower bound is universal, in that it holds as long as the entries of the $\boldsymbol{\zeta}(k)$ vectors are i.i.d. with mean zero, variance 1, and finite fourth moment. The proof of the upper bound $\limsup_{n \to \infty} V^n \leq V^{\infty}$ is more delicate, relying on dynamic programming principles and a priori bounds obtained from a coupling procedure involving the F\"ollmer drift, which makes explicit use of the Gaussian structure. In addition to our main convergence result, we provide some analysis and asymptotics for the limiting mean-field control problem.

math.PR

Mean field control with stopping

We study a high-dimensional stochastic optimization problem which features both control and stopping. In particular, a central planner steers a large population of particles, and can also remove particles at any time by paying a penalty. In the limit, we obtain a mean field control problem with discontinuous dynamics, in the sense that the controlled Fokker-Planck equation can have jumps. The value of the N-particle problem is characterized by a hierarchy of non-linear obstacle problems. The value of the limiting problem, meanwhile, solves an infinite-dimensional quasi-variational inequality (QVI). We introduce a notion of viscosity solution for this QVI, and obtain a comparison principle. Together with various regularity estimates, this comparison principle allows us to characterize the mean field value function as the unique viscosity solution of the QVI, and to establish the convergence of the N-particle value functions to the mean field value function.

math.OC

Unconditional well-posedness of the master equation for monotone mean field games of controls

We establish the first unconditional well-posedness result for the master equation associated with a general class of mean field games of controls. Our analysis covers games with displacement monotone or Lasry--Lions monotone data, as well as those with a small time horizon. By unconditional, we mean that all assumptions are imposed solely at the level of the Lagrangian and the terminal cost. In particular, we do not require any a priori regularity or structural assumptions on the additional fixed-point mappings arising from the control interactions; instead we show that these fixed-point mappings are well-behaved as a consequence of the regularity and the monotonicity of the data. Our approach is bottom-up in nature, unlike most previous results which rely on a generalized method of characteristics. In particular, we build a classical solution of the master equation by showing that the solutions of the corresponding $N$-player Nash systems are compact, in an appropriate sense, and that their subsequential limit points must be solutions to the master equation. Compactness is obtained via uniform-in-$N$ decay estimates for derivatives of the $N$-player value functions. The underlying games are driven by non-degenerate idiosyncratic Brownian noise, and our results allow for the presence of common noise with constant intensity.

math.AP

Error estimates for finite-dimensional approximations of Hamilton-Jacobi-Bellman equations on the Wasserstein space

In this paper, we study a Hamilton-Jacobi-Bellman (HJB) equation set on the Wasserstein space $\mathcal{P}_2(\mathbb{R}^d)$, with a second order term arising from a purely common noise. We do not assume that the Hamiltonian is convex in the momentum variable, which means that we cannot rely on representation formulas coming from mean field control. In this setting, Gangbo, Mayorga, and \'Swi\k{e}ch showed via viscosity solutions methods that the HJB equation on $\mathcal{P}_2(\mathbb{R}^d)$ can be approximated by a sequence of finite-dimensional HJB equations. Our main contribution is to quantify this convergence result. The proof involves a doubling of variables argument, which leverages the Hilbertian approach of P.L. Lions for HJB equations in the Wasserstein space, rather than working with smooth metrics which have been used to obtain similar results in the presence of idiosyncratic noise. In dimension one, our doubling of variables argument is made relatively simply by the rigid structure of one-dimensional optimal transport, but in higher dimension the argument is significantly more complicated, and relies on some estimates concerning the "simultaneous quantization" of probability measures.

math.AP

Mean field control with absorption

In this paper we study a mean field control problem in which particles are absorbed when they reach the boundary of a smooth domain. The value of the N-particle problem is described by a hierarchy of Hamilton-Jacobi equations which are coupled through their boundary conditions. The value function of the limiting problem; meanwhile, solves a Hamilton-Jacobi equation set on the space of sub-probability measures on the smooth domain, i.e. the space of non-negative measures with total mass at most one. Our main contributions are (i) to establish a comparison principle for this novel infinite-dimensional Hamilton-Jacobi equation and (ii) to prove that the value of the N-particle problem converges in a suitable sense towards the value of the limiting problem as N tends to infinity.

math.AP

Quantitative convergence for displacement monotone Mean Field Games of control

In this paper we establish quantitative convergence results for both open and closed-loop Nash equilibria of N-player stochastic differential games in the setting of Mean Field Games of Controls (MFGC), a class of models where interactions among agents occur through both states and controls. Our analysis covers a general class of non-separable Hamiltonians satisfying a displacement monotonicity condition, along with mild regularity and growth conditions at infinity. A major novelty of our work is the rigorous treatment of a nontrivial fixed-point problem on a space of measures, which arises naturally in the MFGC formulation. Unlike prior works that either restrict to separable Hamiltonians - rendering the fixed-point map trivial - or assume convergence or regularity properties of the fixed point map, we develop a detailed structural analysis of this equation and its N-player analogue. This leads to new regularity results for the fixed-point maps and, in turn, to quantitative convergence of open-loop equilibria. We further derive sharp a priori estimates for the N-player Nash system, enabling us to control the discrepancy between open and closed-loop strategies, and thus to conclude the convergence of closed-loop equilibria. Our framework also accommodates common noise in a natural way.

math.PR

A non-asymptotic approach to stochastic differential games with many players under semi-monotonicity

We consider stochastic differential games with a large number of players, with the aim of quantifying the gap between closed-loop, open-loop and distributed equilibria. We show that, under two different semi-monotonicity conditions, the equilibrium trajectories are close when the interactions between the players are weak. Our approach is non-asymptotic in nature, in the sense that it does not make use of any a priori identification of a limiting model, like in mean field game (MFG) theory. The main technical step is to derive bounds on solutions to systems of PDE/FBSDE characterizing the equilibria that are independent of the number of players. When specialized to the mean field setting, our estimates yield quantitative convergence results for both open-loop and closed-loop equilibria without any use of the master equation. In fact, our main bounds hold for games in which interactions are much sparser than those of MFGs, and so we can also obtain some "universality" results for MFGs, in which we show that games governed by dense enough networks converge to the usual MFG limit. Finally, we use our estimates to study a joint vanishing viscosity and large population limit in the setting of displacement monotone games without idiosyncratic noise.

math.PR

Quantitative convergence for mean field control with common noise and degenerate idiosyncratic noise

We consider the convergence problem in the setting of mean field control with common noise and degenerate idiosyncratic noise. Our main results establish a rate of convergence of the finite-dimensional value functions $V^N$ towards the mean field value function $U$. In the case that the idiosyncratic noise is constant (but possibly degenerate), we obtain the rate $N^{-1/(d+7)}$, which is close to the conjectured optimal rate $N^{-1/d}$, and improves on the existing literature even in the non-degenerate setting. In the case that the idiosyncratic noise can be both non-constant and degenerate, the argument is more complicated, and we instead find the rate $N^{-1/(3d + 19)}$. Our proof strategy builds on the one initiated in [Daudin, Delarue, Jackson - JFA, 2024] in the case of non-degenerate idiosyncratic noise and zero common noise, which consists of approximating $U$ by more regular functions which are almost subsolutions of the infinite-dimensional Hamilton-Jacobi equation solved by $U$. Because of the different noise structure, several new steps are necessary in order to produce an appropriate mollification scheme. In addition to our main convergence results, we investigate the case of zero idiosyncratic noise, and show that sharper results can be obtained there by purely control-theoretic arguments. We also provide examples to demonstrate that the value function is sensitive to the choice of admissible controls in the zero noise setting.

math.OC

Concentration bounds for stochastic systems with singular kernels

This note is concerned with weakly interacting stochastic particle systems with possibly singular pairwise interactions. In this setting, we observe a connection between entropic propagation of chaos and exponential concentration bounds for the empirical measure of the system. In particular, we establish a variational upper bound for the probability of a certain rare event, and then use this upper bound to show that "controlled" entropic propagation of chaos implies an exponential concentration bound for the empirical measure. This connection allows us to infer concentration bounds for a class of singular stochastic systems through a simple adaptation of the arguments developed in Jabin and Wang (2018).

math.PR

Sharp convergence rates for mean field control in the region of strong regularity

We study the convergence problem for mean field control, also known as optimal control of McKean-Vlasov dynamics. We assume that the data is smooth but not convex, and thus the limiting value function $\mathcal{U} :[0,T] \times \mathcal{P}_2(\mathbb{R}^d) \to \mathbb{R}$ is Lipschitz, but may not be differentiable. In this setting, the first and last named authors recently identified an open and dense subset $\mathcal{O}$ of $[0,T] \times \mathcal{P}_2(\mathbb{R}^d)$ on which $\mathcal{U}$ is $\mathcal{C}^1$ and solves the relevant infinite-dimensional Hamilton-Jacobi equation in a classical sense. In the present paper, we use these regularity results, and some non-trivial extensions of them, to derive sharp rates of convergence. In particular, we show that the value functions for the $N$-particle control problems converge towards $\mathcal{U}$ with a rate of $1/N$, uniformly on subsets of $\mathcal{O}$ which are compact in the $p$-Wasserstein space for some $p > 2$. A similar result is also established at the level of the optimal feedback controls. The rate $1/N$ is the optimal rate in this setting even if $\mathcal{U}$ is smooth, while, in general, the optimal global rate of convergence is known to be slower than $1/N$. Thus our results show that the rate of convergence is faster inside of $\mathcal{O}$ than it is outside. As a consequence of the convergence of the optimal feedbacks, we obtain a concentration inequality for optimal trajectories of the $N$-particle problem started from i.i.d. initial conditions.

math.OC

Well-posedness of Hamilton-Jacobi equations in the Wasserstein space: non-convex Hamiltonians and common noise

We establish the well-posedness of viscosity solutions for a class of semi-linear Hamilton-Jacobi equations set on the space of probability measures on the torus. In particular, we focus on equations with both common and idiosyncratic noise, and with Hamiltonians which are not necessarily convex in the momentum variable. Our main results show (i) existence, (ii) the comparison principle (and hence uniqueness), and (iii) the convergence of finite-dimensional approximations for such equations. Our proof strategy for the comparison principle is to first use a mix of existing techniques (especially a change of variables inspired by \cite{bayraktar2023} to deal with the common noise) to prove a ``partial comparison" result, i.e. a comparison principle which holds when either the subsolution or the supersolution is Lipschitz with respect to a certain very weak metric. Our main innovation is then to develop a strategy for removing this regularity assumption. In particular, we use some delicate estimates for a sequence of finite-dimensional PDEs to show that under certain conditions there \textit{exists} a viscosity solution which is Lipschitz with respect to the relevant weak metric. We then use this existence result together with a mollification procedure to establish a full comparison principle, i.e. a comparison principle which holds even when the subsolution under consideration is just upper semi-continuous (with respect to the weak topology) and the supersolution is just lower semi-continuous. We then apply these results to mean-field control problems with common noise and zero-sum games over the Wasserstein space.

math.AP

On the Optimal Rate for the Convergence Problem in Mean Field Control

The goal of this work is to obtain optimal rates for the convergence problem in mean field control. Our analysis covers cases where the solutions to the limiting problem may not be unique nor stable. Equivalently the value function of the limiting problem might not be differentiable on the entire space. Our main result is then to derive sharp rates of convergence in two distinct regimes. When the data is sufficiently regular, we obtain rates proportional to $N^{-1/2}$, with $N$ being the number of particles. When the data is merely Lipschitz and semi-concave with respect to the first Wasserstein distance, we obtain rates proportional to $N^{-2/(3d+6)}$. Noticeably, the exponent $2/(3d+6)$ is close to $1/d$, which is the optimal rate of convergence for uncontrolled particle systems driven by data with a similar regularity. The key argument in our approach consists in mollifying the value function of the limiting problem in order to produce functions that are almost classical sub-solutions to the limiting Hamilton-Jacobi equation (which is a PDE set on the space of probability measures). These sub-solutions can be projected onto finite dimensional spaces and then compared with the value functions associated with the particle systems. In the end, this comparison is used to prove the most demanding bound in the estimates. The key challenge therein is thus to exhibit an appropriate form of mollification. We do so by employing sup-convolution within a convenient functional Hilbert space. To make the whole easier, we limit ourselves to the periodic setting. We also provide some examples to show that our results are sharp up to some extent.

math.OC

Quantitative convergence for displacement monotone mean field games with controlled volatility

We study the convergence problem for mean field games with common noise and controlled volatility. We adopt the strategy recently put forth by Lauri\`ere and the second author, using the maximum principle to recast the convergence problem as a question of ``forward-backward propagation of chaos", i.e (conditional) propagation of chaos for systems of particles evolving forward and backward in time. Our main results show that displacement monotonicity can be used to obtain this propagation of chaos, which leads to quantitative convergence results for open-loop Nash equilibria for a class of mean field games. Our results seem to be the first (quantitative or qualitative) which apply to games in which the common noise is controlled. The proofs are relatively simple, and rely on a well-known technique for proving well-posedness of FBSDEs which is combined with displacement monotonicity in a novel way. To demonstrate the flexibility of the approach, we also use the same arguments to obtain convergence results for a class of infinite horizon discounted mean field games.

math.PR

Approximately optimal distributed stochastic controls beyond the mean field setting

We study high-dimensional stochastic optimal control problems in which many agents cooperate to minimize a convex cost functional. We consider both the full-information problem, in which each agent observes the states of all other agents, and the distributed problem, in which each agent observes only its own state. Our main results are sharp non-asymptotic bounds on the gap between these two problems, measured both in terms of their value functions and optimal states. Along the way, we develop theory for distributed optimal stochastic control in parallel with the classical setting, by characterizing optimizers in terms of an associated stochastic maximum principle and a Hamilton-Jacobi-type equation. By specializing these results to the setting of mean field control, in which costs are (symmetric) functions of the empirical distribution of states, we derive the optimal rate for the convergence problem in the displacement convex regime.

math.PR

On quasilinear parabolic systems and FBSDEs of quadratic growth

Using probabilistic methods, we establish a-priori estimates for two classes of quasilinear parabolic systems of partial differential equations (PDEs). We treat in particular the case of a nonlinearity which has quadratic growth in the gradient of the unknown. As a result of our estimates, we obtain the existence of classical solutions of the PDE system. From this, we infer the existence of solutions to a corresponding class of forward-backward stochastic differential equations.

math.PR

An algebraic convergence rate for the optimal control of McKean-Vlasov dynamics

We establish an algebraic rate of convergence in the large number of players limit of the value functions of N-particle stochastic control problems towards the value function of the corresponding McKean-Vlasov problem also known as mean field control. The rate is obtained in the presence of both idiosyncratic and common noises and in a setting where the value function for the McKean-Vlasov problem need not be smooth. Our approach relies crucially on uniform in N Lipschitz and semi-concavity estimates for the N-particle value functions as well as a certain concentration inequality.

math.OC

The reverse H\"older inequality for matrix-valued stochastic exponentials and applications to quadratic BSDE systems

In this paper, we study the connections between three concepts - the reverse H\"older inequality for matrix-valued martingales, the well-posedness of linear BSDEs with unbounded coefficients, and the well-posedness of quadratic BSDE systems. In particular, we show that a linear BSDE with bmo (bounded mean oscillation) coefficients is well-posed if and only if the stochastic exponential of a related matrix-valued martingale satisfies a reverse H\"older inequality. Furthermore, we give structural conditions under which these two equivalent conditions are satisfied. Finally, we apply our results on linear equations to obtain global well-posedness results for two new classes of non-Markovian quadratic BSDE systems with special structure.

math.PR

A new monotonicity condition for ergodic BSDEs and ergodic control with super-quadratic Hamiltonians

We establish the existence (and in an appropriate sense uniqueness) of Markovian solutions for ergodic BSDEs under a novel monotonicity condition. Our monotonicity condition allows us to prove existence even when the driver f has arbitrary (in particular super-quadratic) growth in z, which reveals an interesting trade-off between monotonicity and growth for ergodic BSDEs. The technique of proof is to establish a probabilistic representation of the derivative of the Markovian solution, and then use this representation to obtain a-priori estimates. Our study is motivated by applications to ergodic control, and we use our existence result to prove the existence of optimal controls for a class of ergodic control problems with potentially super-quadratic Hamiltonians. We also treat a class of drivers coming from the construction of forward performance processes, and interpret our monotonicity condition in this setting.

math.PR