arXiv subjects

Huyên Pham

Publications and source records attributed to Huyên Pham.

At least 19 recordsLinked to original sources

arXiv2026

Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control

We develop a model-free policy gradient method for discrete-time mean-field control (MFC). In MFC, the policy affects the objective both through the controlled dynamics and through the population distribution. Standard REINFORCE estimators capture the first effect but not the second. We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution. The method applies to both finite and continuous state spaces. In finite state spaces, we perturb the population distribution directly on the probability simplex through a convex combination of the current population weights and random weights. In continuous state spaces, we project the population distribution onto the manifold of Gaussian mixtures, and then randomize it via a transport map that ensures the perturbed law remains within this manifold. We prove consistency of the perturbed objective and gradient as the perturbation vanishes, and derive bias and mean-square error bounds for the resulting sample-based gradient estimator. Numerical experiments on several MFC benchmarks show that Transport REINFORCE improves over standard REINFORCE.

math.OC↗

arXiv2026

Stochastic Maximum Principle for McKean-Vlasov Control with Discrete Path Dependence

We study a class of McKean-Vlasov control problems with discrete path dependence. The coefficients and cost functional may depend on finitely many past values of the controlled state, observed at fixed deterministic times, and on their joint law. We establish well-posedness of the controlled state equation and derive necessary and sufficient optimality conditions through a stochastic Pontryagin maximum principle. The adjoint process is characterized by a backward stochastic differential equation with jumps at fixed observation times. Each jump represents the conditional sensitivity of future costs with respect to the corresponding observed state and its distribution. In the linear-quadratic case, we prove global solvability of the resulting forward-backward system by combining a continuation argument with a mean-field Riccati reduction. We finally discuss an application to time-series generation, where the terminal cost is given by a kernel-based discrepancy between the law of the sampled path and a target path distribution.

math.OC↗

arXiv2026

Deep-MKV-TS: Path-Dependent McKean--Vlasov Control for Financial Time Series Generation

We introduce Deep-MKV-TS, a path-dependent McKean-Vlasov framework for financial scenario generation. The stochastic dynamics are chosen by matching selected path and volatility features of generated scenarios to those observed in the data. Starting from an interpretable reference model, Deep-MKV-TS preserves the reference drift and adjusts its volatility, while a regularization penalty limits unnecessary departures from the calibrated dynamics. We solve the resulting control problem using a neural, sample-based implementation of the stochastic maximum principle. We validate the method against an exactly computable oracle. On Heston and Heston-mixture models, Deep-MKV-TS substantially reduces path-dependent and volatility-related deficiencies of the reference model. In delayed-volatility experiments, the correction remains effective as the forecasting horizon increases, while direct training becomes less reliable. On held-out intraday equity-index futures, the corrected model improves conditional forecasts relative to the reference and reaches a level of performance comparable to flexible generative and historical baselines. The resulting scenarios also support greater exposure than the reference under a fixed drawdown-risk target. These results show that path-dependent McKean-Vlasov control can enrich an interpretable reference model without replacing it.

q-fin.CP↗

arXiv2026

Policy Gradient Learning for Distributionally Robust Markov Decision Processes under Wasserstein Ambiguity

We study finite-horizon Markov decision processes under distributional uncertainty in the transition kernels and develop a policy-gradient framework for Wasserstein distributionally robust control. Ambiguity is modeled by Wasserstein balls of common radius centered at state--action-dependent nominal transition kernels, leading to a max--min problem over randomized policies and admissible transition laws. Because the worst-case transition law depends implicitly on the policy parameters, the standard policy-gradient argument does not apply directly. We address this difficulty by combining the dynamic programming recursion with Wasserstein duality and a primal envelope argument. In general, the right and left directional derivatives of the one-step worst-case value are obtained by taking the minimum or maximum expected downstream value derivative over the set of worst-case transition laws. In finite state--action spaces, this set is characterized through the optimal face of a transport linear program, yielding an exact directional-derivative recursion. Under the required stability conditions and uniqueness of the active dual and transport optimizers, the derivative becomes linear in the policy perturbation and admits an explicit vector valued policy-gradient recursion. Building on this representation, we propose a robust actor--critic implementation and evaluate it on benchmark examples.

math.OC↗

arXiv2026

Actor-Critic Learning for Extended Mean Field Control with Deterministic Policies

This paper develops a model-free reinforcement learning framework for continuous--time extended mean field control problems, where both the dynamics and reward may depend on the joint distribution of states and controls. We adopt deterministic feedback policies, under which the state--action distribution is induced directly as a push--forward of the state law. This avoids optimization over stochastic kernels and bypasses key limitations of existing approaches in extended mean field settings. We first establish a model--free sensitivity formula for parameterized McKean--Vlasov dynamics and use it to derive a deterministic policy gradient formula expressed through an advantage--rate function on the Wasserstein space. We then refine this formula by introducing local value and advantage--rate representations that depend on the state, action, and joint state--action distribution, yielding a policy gradient that includes both action derivatives and measure--derivative terms with respect to the control distribution. These characterizations lead to a martingale--based learning principle and motivate a continuous--time deep deterministic policy gradient algorithm combining particle approximations, measure--dependent neural networks, temporal--difference learning, and exploration in either action or parameter space. Numerical experiments on stochastic Cucker--Smale consensus control and optimal liquidation with trade crowding demonstrate the efficiency, stability, and robustness of the proposed method, including problems with explicit dependence on the control distribution.

math.OC↗

arXiv2026

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging. Existing deep learning solvers often rely on repeated automatic differentiation to evaluate differential operators, which can cause instability and amplify derivative errors in high dimensions, while probabilistic methods based on stochastic representations require explicit knowledge of the data-generating dynamics and therefore do not apply to black-box environments. We introduce two types of simulators as data-generating mechanisms, and take a ``representing-then-learning" approach that learns the solutions and their derivatives under settings where the underlying PDE operators are accessible only through simulations and pointwise evaluations. Our representation of derivatives relies on the zeroth-order derivative (ZOD) estimators derived from perturbed Monte Carlo trajectories. This fully model-free approach generates targets for the gradient and Hessian networks using only function evaluations. We provide a statistical learning analysis of the proposed approach, including a bias--variance tradeoff for ZODs. Assuming a standard contraction property of the underlying operator, we establish a non-asymptotic error bound that decomposes the total error into discretization error, approximation error, statistical error, and ZOD bias. Crucially, we derive the sample complexity of the learned representations in (weighted) Sobolev space, characterizing the error up to second-order derivatives. Numerical experiments illustrate the competitive performance of the method in moderate and high dimensions.

cs.LG↗

arXiv2026

Bridging Schrödinger and Bass: A Semimartingale Optimal Transport Problem with Diffusion Control

We study a semimartingale optimal transport problem interpolating between the Schrödinger bridge and the stretched Brownian motion associated with the Bass solution of the Skorokhod embedding problem. The cost combines an entropy term on the drift with a quadratic penalization of the diffusion coefficient, leading to a stochastic control problem over drift and volatility. We establish a complete duality theory for this problem, despite the lack of coercivity in the diffusion component. In particular, we prove strong duality and dual attainment, and derive an equivalent reduced dual formulation in terms of a variational problem over terminal potentials. Optimal solutions are characterized by a coupled Schrödinger-Bass bridge system, involving a backward heat potential and a transport map given by the gradient of a $β$-convex function. This system interpolates between the classical Schrödinger system and the Bass martingale transport. Our results furnish a unified framework encompassing entropic and martingale optimal transport, and yield a variational foundation for data-driven diffusion models.

math.PR↗

arXiv2026

Learning Generative Dynamics with Soft Law Constraints: A McKean-Vlasov FBSDE Approach

We propose a generative framework for learning stochastic dynamics from endpoint and intermediate distributional observations. The method formulates generation as a McKean-Vlasov control problem in which terminal and time-marginal laws are enforced through soft energy constraints. The associated optimality system is a forward-backward stochastic differential equation (FBSDE) whose backward component receives a continuous drift induced by the marginal law penalties. This provides a principled alternative to hard interpolation or optimal transport maps between observed distributions: the model learns a stochastic path law whose dynamics remain globally coupled through the mean-field objective. We derive the reduced FBSDE system for quadratic control cost and constant diffusion, connecting terminal and marginal law flat derivatives to score-like training signals. The resulting neural solver is evaluated on low-dimensional distributional benchmarks, where it recovers smooth stochastic paths matching prescribed marginal laws. In a higher-dimensional ALAE latent space, endpoint supervision is used as a qualitative stress test for transporting non-smiling faces toward smiling ones in a pretrained representation. We then use articulated human motion as a structured high-dimensional case study on a curated AMASS low-to-high position dataset, using SMPL-H pose sequences and reduced pose representations. The experiments show that soft marginal law constraints can produce coherent stochastic trajectories whose intermediate distributions follow the observed evolution of human motion. The code is available at https://github.com/murex/deep-mkv-gen/tree/main.

math.OC↗

arXiv2026

Direct Estimation of Schrödinger Bridge Time-Series Drifts: Finite-Sample, Asymptotic, and Adaptive Guarantees

We study nonparametric estimation of Schrödinger bridge (SB) drifts from i.i.d.\ data observed on a single time interval. Starting from the conditional-ratio form of the Schrödinger bridge time-series (SBTS) drift formula, we analyze a direct Nadaraya--Watson plug-in estimator built from kernelized numerator and denominator terms. Unlike recent SB analyses based on entropic-OT potentials, Sinkhorn iterations, or iterative bridge solvers, our approach works directly at the drift level and isolates \emph{statistical error} from optimization, approximation, and discretization error. Under Hölder regularity, a marginal-density floor, and bounded support, we prove a uniform non-asymptotic bound for admissible bandwidth pairs, a pointwise CLT under genuine undersmoothing, and an adaptive bandwidth selector satisfying an oracle inequality. We also prove a pivot-local minimax lower bound which, through an explicit uniform pivot, yields a global minimax lower bound under transparent compatibility conditions; hence the adaptive selector is minimax-rate optimal up to logarithmic factors. Synthetic experiments provide theorem-targeted diagnostics for finite-sample scaling, Gaussian approximation, and adaptive behavior.

math.ST↗

arXiv2026

LightSBB-M: Bridging Schrödinger and Bass for Generative Diffusion Modeling

The Schrodinger Bridge and Bass (SBB) formulation, which jointly controls drift and volatility, is an established extension of the classical Schrodinger Bridge (SB). Building on this framework, we introduce LightSBB-M, an algorithm that computes the optimal SBB transport plan in only a few iterations. The method exploits a dual representation of the SBB objective to obtain analytic expressions for the optimal drift and volatility, and it incorporates a tunable parameter beta greater than zero that interpolates between pure drift (the Schrodinger Bridge) and pure volatility (Bass martingale transport). We show that LightSBB-M achieves the lowest 2-Wasserstein distance on synthetic datasets against state-of-the-art SB and diffusion baselines with up to 32 percent improvement. We also illustrate the generative capability of the framework on an unpaired image-to-image translation task (adult to child faces in FFHQ). These findings demonstrate that LightSBB-M provides a scalable, high-fidelity SBB solver that outperforms existing SB and diffusion baselines across both synthetic and real-world generative tasks. The code is available at https://github.com/alexouadi/LightSBB-M.

cs.LG↗

arXiv2026

Discretization error from regularized Reinforcement Learning to continuous-time stochastic control

This paper establishes a rigorous connection between regularized discrete-time reinforcement learning (RL) and continuous-time stochastic optimal control. Specifically, classical RL algorithms are typically solving a regularized discrete-time Bellman equation. We study the discretization error, namely, the gap between the optimal policy induced by the regularized discrete-time Bellman equation and the true optimal feedback control of the underlying continuous-time stochastic control problem. By deriving quantitative convergence rates for this gap, we provide a rigorous foundation for understanding the stability and implementation of exploratory RL policies in stochastic continuous-time environments.

math.OC↗

arXiv2026

SBBTS: A Unified Schrödinger-Bass Framework for Synthetic Financial Time Series

We study the problem of generating synthetic time series that reproduce both marginal distributions and temporal dynamics, a central challenge in financial machine learning. Existing approaches typically fail to jointly model drift and stochastic volatility, as diffusion-based methods fix the volatility while martingale transport models ignore drift. We introduce the Schrödinger-Bass Bridge for Time Series (SBBTS), a unified framework that extends the Schrödinger-Bass formulation to multi-step time series. The method constructs a diffusion process that jointly calibrates drift and volatility and admits a tractable decomposition into conditional transport problems, enabling efficient learning. Numerical experiments on the Heston model demonstrate that SBBTS accurately recovers stochastic volatility and correlation parameters that prior SchrödingerBridge methods fail to capture. Applied to S&P 500 data, SBBTS-generated synthetic time series consistently improve downstream forecasting performance when used for data augmentation, yielding higher classification accuracy and Sharpe ratio compared to real-data-only training. These results show that SBBTS provides a practical and effective framework for realistic time series generation and data augmentation in financial applications.

cs.LG↗

arXiv2026

Learning operators on labelled conditional distributions with applications to mean field control of non exchangeable systems

We study the approximation of operators acting on probability measures on a product space with prescribed marginal. Let $I$ be a label space endowed with a reference measure $λ$, and define $\cal M_λ$ as the set of probability measures on $I\times \mathbb{R}^d$ with first marginal $λ$. By disintegration, elements of $\cal M_λ$ correspond to families of labeled conditional distributions. Operators defined on this constrained measure space arise naturally in mean-field control problems with heterogeneous, non-exchangeable agents. Our main theoretical result establishes a universal approximation theorem for continuous operators on $\cal M_λ$. The proof combines cylindrical approximations of probability measures with DeepONet-type branch-trunk neural architecture, yielding finite-dimensional representations of such operators. We further introduce a sampling strategy for generating training measures in $\cal M_λ$, enabling practical learning of such conditional mean-field operators. We apply the method to the numerical resolution of mean-field control problems with heterogeneous interactions, thereby extending previous neural approaches developed for homogeneous (exchangeable) systems. Numerical experiments illustrate the accuracy and computational effectiveness of the proposed framework.

math.OC↗

arXiv2026

Schrödinger bridges with jumps for time series generation

We study generative modeling for time series using entropic optimal transport and the Schrödinger bridge (SB) framework, with a focus on applications in finance and energy modeling. Extending the diffusion-based approach of Hamdouche, Henry-Labordère, Pham, 2023, we introduce a jump-diffusion Schrödinger bridge model that allows for discontinuities in the generative dynamics. Starting from a Schrödinger bridge entropy minimization problem, we reformulate the task as a stochastic control problem whose solution characterizes the optimal controlled jump-diffusion process. When sampled on a fixed time grid, this process generates synthetic time series matching the joint distributions of the observed data. The model is fully data-driven, as both the drift and the jump intensity are learned directly from the data. We propose practical algorithms for training, sampling, and hyperparameter calibration. Numerical experiments on simulated and real datasets, including financial and energy time series, show that incorporating jumps substantially improves the realism of the generated data, in particular by capturing abrupt movements, heavy tails, and regime changes that diffusion-only models fail to reproduce. Comparisons with state-of-the-art generative models highlight the benefits and limitations of the proposed approach.

q-fin.MF↗

arXiv2026

Model-free policy gradient for discrete-time mean-field control

We study model-free policy learning for discrete-time mean-field control (MFC) problems with finite state space and compact action space. In contrast to the extensive literature on value-based methods for MFC, policy-based approaches remain largely unexplored due to the intrinsic dependence of transition kernels and rewards on the evolving population state distribution, which prevents the direct use of likelihood-ratio estimators of policy gradients from classical single-agent reinforcement learning. We introduce a novel perturbation scheme on the state-distribution flow and prove that the gradient of the resulting perturbed value function converges to the true policy gradient as the perturbation magnitude vanishes. This construction yields a fully model-free estimator based solely on simulated trajectories and an auxiliary estimate of the sensitivity of the state distribution. Building on this framework, we develop MF-REINFORCE, a model-free policy gradient algorithm for MFC, and establish explicit quantitative bounds on its bias and mean-squared error. Numerical experiments on representative mean-field control tasks demonstrate the effectiveness of the proposed approach.

math.OC↗

arXiv2026

Non-Exchangeable Mean Field Markov Decision Processes with common noise : from Bellman equation to quantitative propagation of chaos

We study infinite-horizon Markov Decision Processes (MDPs) with a continuum of heterogeneous agents interacting through a common noise, without assuming exchangeability. We introduce the framework of Conditional Non-Exchangeable Mean Field MDPs (CNEMF-MDPs) in both a strong formulation and a label-state formulation. We establish the equivalence between these two formulations by showing that the control problem can be lifted to a standard MDP defined on the Wasserstein space of probability measures over the product of the label and state spaces. Here, the label space represents agent heterogeneity, the state space is the individual state space, and a fixed distribution specifies the population of agent labels. Within this framework, we characterize the value function as the unique fixed point of an appropriate Bellman operator acting on this Wasserstein space. Our second contribution is a quantitative analysis of the propagation of chaos for this non-exchangeable setting with common noise. We derive sharp finite-population bounds by comparing the Bellman operator of the finite N-agent MDP, defined on the the N-fold product of the state space, with its infinite-agent counterpart. This comparison yields explicit constructions of near-optimal policies for the N-agent system from epsilon-optimal policies of the limiting CNEMF-MDP.

math.OC↗

arXiv2026

A PDE Derivation of the Schrödinger--Bass Bridge

This short paper announces the main results of \cite{SBB2026}, where the Schrödinger--Bass Bridge (SBB) problem is introduced and studied in full generality. Here we provide a direct PDE derivation of the SBB system in dimension one, showing how the optimal coupling problem that interpolates between the classical Schrödinger bridge and the Bass martingale transport can be solved explicitly via Legendre transforms and the heat equation. A key insight is that the optimal SBB process is a Stretched Schrödinger Bridge: the composition of a monotone transport map with a Schrödinger bridge. This extends the stretched Brownian motion representation of Bass martingales to the semimartingale setting and provides a unified framework that recovers both the Sinkhorn algorithm (in the limit $β\to \infty$) and the Bass construction (as $β\to 0$). We refer to \cite{SBB2026} for complete proofs, the multidimensional setting, strong duality, dual attainment, and further developments.

math.PR↗

arXiv2026

Optimal control of McKean-Vlasov systems under partial observation and hidden Markov switching

We study a class of mean-field control problems under partial observation. The controlled dynamics are of McKean-Vlasov type and are subject to regime switching driven by a hidden Markov chain. The observation process depends on the control and on the joint distribution of the state and control, which prevents the direct application of standard filtering techniques. The main contribution of this paper is to show how this distribution dependence can be handled within a change-of-probability framework, leading to a well-posed separated control problem. We derive a Zakai equation with a specific structure for the unnormalized filter, and show that the corresponding value function satisfies a dynamic programming principle. This yields a Bellman equation posed on a convex subset of a Wasserstein space, characterizing the optimal control problem under partial observation.

math.OC↗