arXiv subjects

Huyên Pham

Publications and source records attributed to Huyên Pham.

At least 37 records · Page 2Linked to original sources

arXiv2025

Linear-quadratic optimal control for non-exchangeable mean-field SDEs and applications to systemic risk

We study the linear-quadratic control problem for a class of non-exchangeable mean-field systems, which model large populations of heterogeneous interacting agents. We explicitly characterize the optimal control in terms of a new infinite-dimensional system of Riccati equations, for which we establish existence and uniqueness. To illustrate our results, we apply this framework to a systemic risk model involving heterogeneous banks, demonstrating the impact of agent heterogeneity on optimal risk mitigation strategies.

math.OC↗

arXiv2025

Robust time series generation via Schrödinger Bridge: a comprehensive evaluation

We investigate the generative capabilities of the Schrödinger Bridge (SB) approach for time series. The SB framework formulates time series synthesis as an entropic optimal interpolation transport problem between a reference probability measure on path space and a target joint distribution. This results in a stochastic differential equation over a finite horizon that accurately captures the temporal dynamics of the target time series. While the SB approach has been largely explored in fields like image generation, there is a scarcity of studies for its application to time series. In this work, we bridge this gap by conducting a comprehensive evaluation of the SB method's robustness and generative performance. We benchmark it against state-of-the-art (SOTA) time series generation methods across diverse datasets, assessing its strengths, limitations, and capacity to model complex temporal dependencies. Our results offer valuable insights into the SB framework's potential as a versatile and robust tool for time series generation.

cs.LG↗

arXiv2025

Stochastic maximum principle for optimal control problem of non exchangeable mean field systems

We study the Pontryagin maximum principle by deriving necessary and sufficient conditions for a class of optimal control problems arising in non exchangeable mean field systems, where agents interact through heterogeneous and asymmetric couplings. Our analysis leads to a collection of forward-backward stochastic differential equations (FBSDE) of non exchangeable mean field type. Under suitable assumptions, we establish the solvability of this system. As an illustration, we consider the linear-quadratic case, where the optimal control is characterized by an infinite dimensional system of Riccati equations.

math.OC↗

arXiv2024

A randomisation method for mean-field control problems with common noise

We study mean-field control (MFC) problems with common noise using the control randomisation framework, where we substitute the control process with an independent Poisson point process, controlling its intensity instead. To address the challenges posed by the mean-field interactions in this randomisation approach, we reformulate the admissible control as L 0 -valued processes adapted only to the common noise. We then construct the randomised control problem from this reformulated control process, and show its equivalence to the original MFC problem. Thanks to this equivalence, we can represent the value function as the minimal solution to a backward stochastic differential equation (BSDE) with constrained jumps. Finally, using this probabilistic representation, we derive a randomised dynamic programming principle (DPP) for the value function, expressed as a supremum over equivalent probability measures.

math.OC↗

arXiv2024

Trading with propagators and constraints: applications to optimal execution and battery storage

Motivated by optimal execution with stochastic signals, market impact and constraints in financial markets, and optimal storage management in commodity markets, we formulate and solve an optimal trading problem with a general propagator model under linear functional inequality constraints. The optimal control is given explicitly in terms of the corresponding Lagrange multipliers and their conditional expectations, as a solution to a linear stochastic Fredholm equation. We propose a stochastic version of the Uzawa algorithm on the dual problem to construct the stochastic Lagrange multipliers numerically via a stochastic projected gradient ascent, combined with a least-squares Monte Carlo regression step to approximate their conditional expectations. We illustrate our findings on two different practical applications with stochastic signals: (i) an optimal execution problem with an exponential or a power law decaying transient impact, with either a `no-shorting' constraint in the presence of a `sell' signal, a `no-buying' constraint in the presence of a `buy' signal or a stochastic `stop-trading' constraint whenever the exogenous price drops below a specified reference level; (ii) a battery storage problem with instantaneous operating costs, seasonal signals and fixed constraints on both the charging power and the load capacity of the battery.

math.OC↗

arXiv2024

Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise

We consider reinforcement learning (RL) methods for finding optimal policies in linear quadratic (LQ) mean field control (MFC) problems over an infinite horizon in continuous time, with common noise and entropy regularization. We study policy gradient (PG) learning and first demonstrate convergence in a model-based setting by establishing a suitable gradient domination condition.Next, our main contribution is a comprehensive error analysis, where we prove the global linear convergence and sample complexity of the PG algorithm with two-point gradient estimates in a model-free setting with unknown parameters. In this setting, the parameterized optimal policies are learned from samples of the states and population distribution.Finally, we provide numerical evidence supporting the convergence of our implemented algorithms.

math.OC↗

arXiv2024

Mean-field control of non exchangeable systems

We study the optimal control of mean-field systems with heterogeneous and asymmetric interactions. This leads to considering a family of controlled Brownian diffusion processes with dynamics depending on the whole collection of marginal probability laws. We prove the well-posedness of such systems and define the control problem together with its related value function. We next prove a law invariance property for the value function which allows us to work on the set of collections of probability laws. We show that the value function satisfies a dynamic programming principle (DPP) on the flow of collections of probability measures. We also derive a chain rule for a class of regular functions along the flows of collections of marginal laws of diffusion processes. Combining the DPP and the chain rule, we prove that the value function is a viscosity solution of a Bellman dynamic programming equation in a $L^2$-set of Wasserstein space-valued functions.

math.PR↗

arXiv2024

Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching

We propose a comprehensive framework for policy gradient methods tailored to continuous time reinforcement learning. This is based on the connection between stochastic control problems and randomised problems, enabling applications across various classes of Markovian continuous time control problems, beyond diffusion models, including e.g. regular, impulse and optimal stopping/switching problems. By utilizing change of measure in the control randomisation technique, we derive a new policy gradient representation for these randomised problems, featuring parametrised intensity policies. We further develop actor-critic algorithms specifically designed to address general Markovian stochastic control issues. Our framework is demonstrated through its application to optimal switching problems, with two numerical case studies in the energy sector focusing on real options.

math.OC↗

arXiv2024

Mean-field neural networks-based algorithms for McKean-Vlasov control problems *

This paper is devoted to the numerical resolution of McKean-Vlasov control problems via the class of mean-field neural networks introduced in our companion paper [25] in order to learn the solution on the Wasserstein space. We propose several algorithms either based on dynamic programming with control learning by policy or value iteration, or backward SDE from stochastic maximum principle with global or local loss functions. Extensive numerical results on different examples are presented to illustrate the accuracy of each of our eight algorithms. We discuss and compare the pros and cons of all the tested methods.

math.OC↗

arXiv2023

Mean-field neural networks: learning mappings on Wasserstein space

We study the machine learning task for models with operators mapping between the Wasserstein space of probability measures and a space of functions, like e.g. in mean-field games/control problems. Two classes of neural networks, based on bin density and on cylindrical approximation, are proposed to learn these so-called mean-field functions, and are theoretically supported by universal approximation theorems. We perform several numerical experiments for training these two mean-field neural networks, and show their accuracy and efficiency in the generalization error with various test distributions. Finally, we present different algorithms relying on mean-field neural networks for solving time-dependent mean-field problems, and illustrate our results with numerical tests for the example of a semi-linear partial differential equation in the Wasserstein space of probability measures.

math.OC↗

arXiv2023

Actor critic learning algorithms for mean-field control with moment neural networks

We develop a new policy gradient and actor-critic algorithm for solving mean-field control problems within a continuous time reinforcement learning setting. Our approach leverages a gradient-based representation of the value function, employing parametrized randomized policies. The learning for both the actor (policy) and critic (value function) is facilitated by a class of moment neural network functions on the Wasserstein space of probability measures, and the key feature is to sample directly trajectories of distributions. A central challenge addressed in this study pertains to the computational treatment of an operator specific to the mean-field framework. To illustrate the effectiveness of our methods, we provide a comprehensive set of numerical results. These encompass diverse examples, including multi-dimensional settings and nonlinear quadratic mean-field control problems with controlled volatility.

stat.ML↗

arXiv2023

Opinion dynamics in communities with major influencers and implicit social influence via mean-field approximation

We study binary opinion formation in a large population where individuals are influenced by the opinions of other individuals. The population is characterised by the existence of (i) communities where individuals share some similar features, (ii) opinion leaders that may trigger unpredictable opinion shifts in the short term (iii) some degree of incomplete information in the observation of the individual or public opinion processes. In this setting, we study three different approximate mechanisms: common sampling approximation, independent sampling approximation, and, what will be our main focus in this paper, McKean-Vlasov (or mean-field) approximation. We show that all three approximations perform well in terms of different metrics that we introduce for measuring population level and individual level errors. In the presence of a common noise represented by the major influencers opinions processes, and despite the absence of idiosyncratic noises, we derive a propagation of chaos type result. For the particular case of a linear model and particular specifications of the major influencers opinion dynamics, we provide additional analysis, including long term behavior and fluctuations of the public opinion. The theoretical results are complemented by some concrete examples and numerical analysis, illustrating the formation of echo-chambers, the propagation of chaos, and phenomena such as snowball effect and social inertia.42 pages

math.PR↗

arXiv2023

Generative modeling for time series via Schr{ö}dinger bridge

We propose a novel generative model for time series based on Schr{ö}dinger bridge (SB) approach. This consists in the entropic interpolation via optimal transport between a reference probability measure on path space and a target measure consistent with the joint data distribution of the time series. The solution is characterized by a stochastic differential equation on finite horizon with a path-dependent drift function, hence respecting the temporal dynamics of the time series distribution. We can estimate the drift function from data samples either by kernel regression methods or with LSTM neural networks, and the simulation of the SB diffusion yields new synthetic data samples of the time series. The performance of our generative model is evaluated through a series of numerical experiments. First, we test with a toy autoregressive model, a GARCH Model, and the example of fractional Brownian motion, and measure the accuracy of our algorithm with marginal and temporal dependencies metrics. Next, we use our SB generated synthetic samples for the application to deep hedging on real-data sets. Finally, we illustrate the SB approach for generating sequence of images.

math.OC↗

arXiv2023

Actor-Critic learning for mean-field control in continuous time

We study policy gradient for mean-field control in continuous time in a reinforcement learning setting. By considering randomised policies with entropy regularisation, we derive a gradient expectation representation of the value function, which is amenable to actor-critic type algorithms, where the value functions and the policies are learnt alternately based on observation samples of the state and model-free estimation of the population state distribution, either by offline or online learning. In the linear-quadratic mean-field framework, we obtain an exact parametrisation of the actor and critic functions defined on the Wasserstein space. Finally, we illustrate the results of our algorithms with some numerical experiments on concrete examples.

stat.ML↗

arXiv2022

Optimal control of path-dependent McKean-Vlasov SDEs in infinite dimension

We study the optimal control of path-dependent McKean-Vlasov equations valued in Hilbert spaces motivated by non Markovian mean-field models driven by stochastic PDEs. We first establish the well-posedness of the state equation, and then we prove the dynamic programming principle (DPP) in such a general framework. The crucial law invariance property of the value function V is rigorously obtained, which means that V can be viewed as a function on the Wasserstein space of probability measures on the set of continuous functions valued in Hilbert space. We then define a notion of pathwise measure derivative, which extends the Wasserstein derivative due to Lions [41], and prove a related functional It{ô} formula in the spirit of Dupire [24] and Wu and Zhang [51]. The Master Bellman equation is derived from the DPP by means of a suitable notion of viscosity solution. We provide different formulations and simplifications of such a Bellman equation notably in the special case when there is no dependence on the law of the control.

math.OC↗

arXiv2022

A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection

We consider the control of McKean-Vlasov dynamics (or mean-field control) with probabilistic state constraints. We rely on a level-set approach which provides a representation of the constrained problem in terms of an unconstrained one with exact penalization and running maximum or integral cost. The method is then extended to the common noise setting. Our work extends (Bokanowski, Picarelli, and Zidani, SIAM J. Control Optim. 54.5 (2016), pp. 2568--2593) and (Bokanowski, Picarelli, and Zidani, Appl. Math. Optim. 71 (2015), pp. 125--163) to a mean-field setting. The reformulation as an unconstrained problem is particularly suitable for the numerical resolution of the problem, that is achieved from an extension of a machine learning algorithm from (Carmona, Lauri{è}re, arXiv:1908.01613 to appear in Ann. Appl. Prob., 2019). A first application concerns the storage of renewable electricity in the presence of mean-field price impact and another one focuses on a mean-variance portfolio selection problem with probabilistic constraints on the wealth. We also illustrate our approach for a direct numerical resolution of the primal Markowitz continuous-time problem without relying on duality.

math.OC↗

arXiv2022

Itô's formula for flows of measures on semimartingales

We establish Itô's formula along flows of probability measures associated with general semimartingales; this generalizes existing results for flows of measures on Itô processes. Our approach is to first establish Itô's formula for cylindrical functions and then extend it to the general case via function approximation and localization techniques. This general form of Itô's formula enables the derivation of dynamic programming equations and verification theorems for McKean--Vlasov controls with jump diffusions and for McKean--Vlasov mixed regular-singular control problems. It also allows for generalizing the classical relationship between the maximum principle and the dynamic programming principle to the McKean--Vlasov singular control setting, where the adjoint process is expressed in terms of the derivative of the value function with respect to the probability measures.

math.PR↗

arXiv2022

Quantitative propagation of chaos for mean field Markov decision process with common noise

We investigate propagation of chaos for mean field Markov Decision Process with common noise (CMKV-MDP), and when the optimization is performed over randomized open-loop controls on infinite horizon. We first state a rate of convergence of order $M_N^γ$, where $M_N$ is the mean rate of convergence in Wasserstein distance of the empirical measure, and $γ\in (0,1]$ is an explicit constant, in the limit of the value functions of $N$-agent control problem with asymmetric open-loop controls, towards the value function of CMKV-MDP. Furthermore, we show how to explicitly construct $(ε+\mathcal{O}(M_N^γ))$-optimal policies for the $N$-agent model from $ε$-optimal policies for the CMKV-MDP. Our approach relies on sharp comparison between the Bellman operators in the $N$-agent problem and the CMKV-MDP, and fine coupling of empirical measures.

math.OC↗