arXiv subjects

Rene Carmona

Publications and source records attributed to Rene Carmona.

At least 19 recordsLinked to original sources

arXiv2026

Kuramoto Mean Field Game with Intrinsic Frequencies

This paper studies a mean field game formulation of the classical Kuramoto model for synchronization. Our model captures the diversity within the population by considering random intrinsic frequencies, which allows us to study the impact of this heterogeneity on synchronization patterns and stability. Our findings contribute insights into the interplay between intrinsic frequency diversity and synchronization dynamics, offering a more realistic understanding of complex systems. The proposed framework has broad applications ranging from coupled oscillators in physics to social dynamics, and serves as a valuable tool for studying networks with distributed intrinsic frequencies.

math.FA↗

arXiv2026

A Singular Control Problem for Data Center Electricity Cost Minimization

The goal of the paper is to provide a complete analysis of a stochastic control problem when the cost to minimize involves the running maximum of the underlying controlled state process. The model is motivated by the electricity cost minimization of a data center facing coincident peak charges and possibly benefiting from participation in a demand-response program. From a mathematical standpoint, the main challenge comes from the inclusion of the running maximum in the state of the system forcing the original problem into a singular stochastic control problem on a closed wedge in the plane with reflection on parts of the boundary. We prove that the value function is locally Lipschitz continuous in space using a gap-monotonicity estimate for coupled reflected loads and a deterministic total-variation bound for the difference of two running maxima. The dynamic programming principle and viscosity formulation are then derived for ordinary progressively measurable controls. The corresponding HJB equation includes a reflecting Neumann condition on the lower boundary for the load direction as well as an oblique boundary condition on the diagonal part of the boundary of the domain due to the effect of the running-maximum process. We prove existence of a solution in the viscosity sense as well as a comparison theorem providing conditional uniqueness in the corresponding viscosity class. Finally we provide numerical results illustrating the trade-off between the two conflicting incentives comprising the expected cost of the data center.

math.OC↗

arXiv2025

Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control

Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls, and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.

math.OC↗

arXiv2025

Particle Approximation for Conditional Control with Soft Killing

The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a \textit{soft killing} relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.

math.OC↗

arXiv2024

Mimicking and Conditional Control with Hard Killing

We first prove a mimicking theorem (also known as a Markovian projection theorem) for the marginal distributions of an Ito process conditioned to not have exited a given domain. We then apply this new result to the proof of a conjecture of P.L. Lions for the optimal control of conditioned processes.

math.PR↗

arXiv2024

Maximizing On-Bill Savings through Battery Management Optimization

In many power grids, a large portion of the energy costs for commercial and industrial consumers are set with reference to the coincident peak load, the demand during the maximum system-wide peak, and their own maximum peak load, the non-coincident peak load. Coincident-peak based charges reflect the allocation of infrastructure updates to end-users for increased capacity, the amount the grid can handle, and for improvement of the transmission, the ability to transport energy across the network. Demand charges penalize the stress on the grid caused by each consumer's peak demand. Microgrids with a local generator, controllable loads, and/or a battery technology have the flexibility to cut their peak load contributions and thereby significantly reduce these charges. This paper investigates the optimal planning of microgrid technology for electricity bill reduction. The specificity of our approach is the leveraging of a scenario generator engine to incorporate probability estimates of coincident peaks and non-coincident peaks into the optimization problem.

math.OC↗

arXiv2024

Cost Attribution And Risk-Averse Unit Commitment In Power Grids Using Integrated Gradient

This paper introduces a novel approach to addressing uncertainty and associated risks in power system management, focusing on the discrepancies between forecasted and actual values of load demand and renewable power generation. By employing Economic Dispatch (ED) with both day-ahead forecasts and actual values, we derive two distinct system costs, revealing the financial risks stemming from uncertainty. We present a numerical algorithm inspired by the Integrated Gradients (IG) method to attribute the contribution of stochastic components to the difference in system costs. This method, originally developed for machine learning, facilitates the understanding of individual input features' impact on the model's output prediction. By assigning numeric values to represent the influence of variability on operational costs, our method provides actionable insights for grid management. As an application, we propose a risk-averse unit commitment framework, leveraging our cost attribution algorithm to adjust the capacity of renewable generators, thus mitigating system risk. Simulation results on the RTS-GMLC grid demonstrate the efficacy of our approach in improving grid reliability and reducing operational costs.

math.OC↗

arXiv2024

Coincident Peak Prediction for Capacity and Transmission Charge Reduction

Meeting the ever-growing needs of the power grid requires constant infrastructure enhancement. There are two important aspects for a grid ability to ensure continuous and reliable electricity delivery to consumers: capacity, the maximum amount the system can handle, and transmission, the infrastructure necessary to deliver electricity across the network. These capacity and transmission costs are then allocated to the end-users according to the cost causation principle. These charges are computed based on the customer demand on coincident peak (CP) events, time intervals when the system-wide electric load is highest. We tackle the problem of predicting CP events based on actual load and forecast data on the load of different jurisdictions. In particular, we identify two main use cases depending on the availability of a forecast. Our approach generates scenarios and formulates Monte-Carlo estimators for predicting CP-day and exact CP-hour events. Finally, we backtest the prediction performance of strategies with adaptive threshold for the prediction task. This analysis enables us to derive practical implications for load curtailment through Battery Energy Storage System (BESS) solutions.

math.OC↗

arXiv2023

From Nash Equilibrium to Social Optimum and vice versa: a Mean Field Perspective

Mean field games (MFG) and mean field control (MFC) problems have been introduced to study large populations of strategic players. They correspond respectively to non-cooperative or cooperative scenarios, where the aim is to find the Nash equilibrium and social optimum. These frameworks provide approximate solutions to situations with a finite number of players and have found a wide range of applications, from economics to biology and machine learning. In this paper, we study how the players can pass from a non-cooperative to a cooperative regime, and vice versa. The first direction is reminiscent of mechanism design, in which the game's definition is modified so that non-cooperative players reach an outcome similar to a cooperative scenario. The second direction studies how players that are initially cooperative gradually deviate from a social optimum to reach a Nash equilibrium when they decide to optimize their individual cost similar to the free rider phenomenon. To formalize these connections, we introduce two new classes of games which lie between MFG and MFC: $λ$-interpolated mean field games, in which the cost of an individual player is a $λ$-interpolation of the MFG and the MFC costs, and $p$-partial mean field games, in which a proportion $p$ of the population deviates from the social optimum by playing the game non-cooperatively. We conclude the paper by providing an algorithm for myopic players to learn a $p$-partial mean field equilibrium, and we illustrate it on a stylized model.

cs.GT↗

arXiv2023

Non-standard Stochastic Control with Nonlinear Feynman-Kac Costs

We consider the conditional control problem introduced by P.L. Lions in his lectures at the Collège de France in November 2016. In his lectures, Lions emphasized some of the major differences with the analysis of classical stochastic optimal control problems, and in so doing, raised the question of the possible differences between the value functions resulting from optimization over the class of Markovian controls as opposed to the general family of open loop controls. The goal of the paper is to elucidate this quandary and provide elements of response to Lions' original conjecture. First, we justify the mathematical formulation of the conditional control problem by the description of practical model from evolutionary biology. Next, we relax the original formulation by the introduction of \emph{soft} as opposed to hard killing, and using a \emph{mimicking} argument, we reduce the open loop optimization problem to an optimization over a specific class of feedback controls. After proving existence of optimal feedback control functions, we prove a superposition principle allowing us to recast the original stochastic control problems as deterministic control problems for dynamical systems of probability Gibbs measures. Next, we characterize the solutions by forward-backward systems of coupled non-linear Partial Differential Equations (PDEs) very much in the spirit of the Mean Field Game (MFG) systems. From there, we identify a common optimizer, proving the conjecture of equality of the value functions. Finally we illustrate the results by convincing numerical experiments.

math.OC↗

arXiv2022

Synchronization in a Kuramoto Mean Field Game

The classical Kuramoto model is studied in the setting of an infinite horizon mean field game. The system is shown to exhibit both synchronization and phase transition. Incoherence below a critical value of the interaction parameter is demonstrated by the stability of the uniform distribution. Above this value, the game bifurcates and develops self-organizing time homogeneous Nash equilibria. As interactions become stronger, these stationary solutions become fully synchronized. Results are proved by an amalgam of techniques from nonlinear partial differential equations, viscosity solutions, stochastic optimal control and stochastic processes.

math.OC↗

arXiv2022

Optimal Execution with Identity Optionality

This paper investigates the impact of anonymous trading on the agents' strategy in an optimal execution framework. It mainly explores the specificity of order attribution on the Toronto Stock Exchange, where brokers can choose to either trade with their own identity or under a generic anonymous code that is common to all the brokers. We formulate a stochastic differential game for the optimal execution problem of a population of $N$ brokers and incorporate permanent and temporary price impacts for both the identity-revealed and anonymous trading processes. We then formulate the limiting mean-field game of controls with common noise and obtain a solution in closed-form via the probabilistic approach for the Almgren-Chris price impact framework. Finally, we perform a sensitivity analysis to explore the impact of the model parameters on the optimal strategy.

q-fin.MF↗

arXiv2022

Joint Stochastic Model for Electric Load, Solar and Wind Power at Asset Level and Monte Carlo Scenario GenerationRené Carmona \& Xinshuo Yang

For the purpose of Monte Carlo scenario generation, we propose a graphical model for the joint distribution of wind power and electricity demand in a given region. To conform with the practice in the electric power industry, we assume that point forecasts are provided exogenously, and concentrate on the modeling of the deviations from these forecasts instead of modeling the actual quantities of interest. We find that the marginal distributions of these deviations can have heavy tails, feature which we need to handle before fitting a graphical Gaussian model to the data. We estimate covariance and precision matrices using an extension of the graphical LASSO procedure which allows us to identify temporal and geographical (conditional) dependencies in the form of separate dependence graphs. We implement our algorithm on data made available by NREL, and we confirm that the dependencies identified by the algorithm are consistent with the relative locations of the production assets and the geographical load zones over which the data were collected.

stat.AP↗

arXiv2022

Mean Field Model for an Advertising Competition in a Duopoly

In this study, we analyze an advertising competition in a duopoly. We consider two different notions of equilibrium. We model the companies in the duopoly as major players, and the consumers as minor players. In our first game model we identify Nash Equilibria (NE) between all the players. Next we frame the model to lead to the search for Multi-Leader-Follower Nash Equilibria (MLF-NE). This approach is reminiscent of Stackelberg games in the sense that the major players design their advertisement policies assuming that the minor players are rational and settle in a Nash Equilibrium among themselves. This rationality assumption reduces the competition between the major players to a 2-player game. After solving these two models for the notions of equilibrium, we analyze the similarities and differences of the two different sets of equilibria.

math.OC↗

arXiv2021

Mean Field Models to Regulate Carbon Emissions in Electricity Production

The most serious threat to ecosystems is the global climate change fueled by the uncontrolled increase in carbon emissions. In this project, we use mean field control and mean field game models to analyze and inform the decisions of electricity producers on how much renewable sources of production ought to be used in the presence of a carbon tax. The trade-off between higher revenues from production and the negative externality of carbon emissions is quantified for each producer who needs to balance in real time reliance on reliable but polluting (fossil fuel) thermal power stations versus investing in and depending upon clean production from uncertain wind and solar technologies. We compare the impacts of these decisions in two different scenarios: 1) the producers are competitive and hopefully reach a Nash Equilibrium; 2) they cooperate and reach a Social Optimum. We first prove that both problems have a unique solution using forward-backward systems of stochastic differential equations. We then illustrate with numerical experiments the producers' behavior in each scenario. We further introduce and analyze the impact of a regulator in control of the carbon tax policy, and we study the resulting Stackelberg equilibrium with the field of producers.

math.OC↗

arXiv2021

Finite State Graphon Games with Applications to Epidemics

We consider a game for a continuum of non-identical players evolving on a finite state space. Their heterogeneous interactions are represented by a graphon, which can be viewed as the limit of a dense random graph. The player's transition rates between the states depend on their own control and the interaction strengths with the other players. We develop a rigorous mathematical framework for this game and analyze Nash equilibria. We provide a sufficient condition for a Nash equilibrium and prove existence of solutions to a continuum of fully coupled forward-backward ordinary differential equations characterizing equilibria. Moreover, we propose a numerical approach based on machine learning tools and show experimental results on different applications to compartmental models in epidemiology.

math.OC↗

arXiv2021

Stochastic Graphon Games: II. The Linear-Quadratic Case

In this paper, we analyze linear-quadratic stochastic differential games with a continuum of players interacting through graphon aggregates, each state being subject to idiosyncratic Brownian shocks. The major technical issue is the joint measurability of the player state trajectories with respect to samples and player labels, which is required to compute for example costs involving the graphon aggregate. To resolve this issue we set the game in a Fubini extension of a product probability space. We provide conditions under which the graphon aggregates are deterministic and the linear state equation is uniquely solvable for all players in the continuum. The Pontryagin maximum principle yields equilibrium conditions for the graphon game in the form of a forward-backward stochastic differential equation, for which we establish existence and uniqueness. We then study how graphon games approximate games with finitely many players over graphs with random weights. We illustrate some of the results with a numerical example.

math.PR↗

arXiv2021

Optimal incentives to mitigate epidemics: a Stackelberg mean field game approach

Motivated by models of epidemic control in large populations, we consider a Stackelberg mean field game model between a principal and a mean field of agents evolving on a finite state space. The agents play a non-cooperative game in which they can control their transition rates between states to minimize an individual cost. The principal can influence the resulting Nash equilibrium through incentives so as to optimize its own objective. We analyze this game using a probabilistic approach. We then propose an application to an epidemic model of SIR type in which the agents control their interaction rate and the principal is a regulator acting with non pharmaceutical interventions. To compute the solutions, we propose an innovative numerical approach based on Monte Carlo simulations and machine learning tools for stochastic optimization. We conclude with numerical experiments by illustrating the impact of the agents' and the regulator's optimal decisions in two models: a basic SIR model with semi-explicit solutions and a more complex model with a larger state space.

math.OC↗