arXiv subjects

Gaozhan Wang

Publications and source records attributed to Gaozhan Wang.

3 recordsLinked to original sources

arXiv2026

Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

We study continuous-time and possibly high-dimensional stochastic control problems where drift coefficients and running reward functions are unknown. Due to these missing model primitives, we take the exploratory, reinforcement learning (RL) framework of Wang, Zariphopoulou, and Zhou(2020) with relaxed controls and entropy regularization. The objective is to develop theoretically grounded, efficient and scalable RL algorithms to learn both the optimal value functions (which also solve the exploratory HJB equation) and optimal exploratory feedback control policies. When the diffusion coefficients do not contain control, we employ probabilistic representations of both the optimal value function and its gradient based on an auxiliary state process depending only on the diffusion part of the original dynamics. With a delicate analysis on some properly defined mappings and their fixed points, this leads to the introduction of our policy iteration algorithms and their convergence. We demonstrate the performance of our algorithms through various numerical examples. Finally, we study a special control-dependent diffusion case where probability representation of the Hessian is called for.

cs.LG↗

arXiv2026

Reinforcement Learning for optimal dividend problem under diffusion model

In this paper, we study the optimal dividend problem under the continuous time diffusion model with the bounded dividend rate from the Reinforcement Learning (RL) perspective. Unlike the standard literature, our main focus will be on numerical algorithms that allow part or all of the system parameters to be unspecified so that the optimal control cannot be explicitly determined. Following the RL literature we introduce the entropy-regularized exploratory control problem, which randomizes the control actions and balances the levels of exploitation and exploration, and carry out a theoretical analysis of the associated Policy Improvement (PI) and Policy Evaluation (PE) devices and the corresponding sequence of the approximating optimal strategies. Specifically, our algorithm will be based on two independent neural networks that approximate the value function and its derivative simultaneously. Such an algorithm, to the best of our knowledge, is new in the context of the optimal dividend problems, and can be effective even for the situation when the premium and/or interest rate is state dependent, hence beyond reach of the standard statistical methods. Some numerical experiments are presented to empirically demonstrate the effectiveness of our RL algorithm.

math.OC↗

arXiv2025

Convergence Analysis for Entropy-Regularized Control Problems: A Probabilistic Approach

In this paper we investigate the convergence of the Policy Iteration Algorithm (PIA) for a class of general continuous-time entropy-regularized stochastic control problems. In particular, instead of employing sophisticated PDE estimates for the iterative PDEs involved in the algorithm (see, e.g., Huang-Wang-Zhou(2025)), we shall provide a simple proof from scratch for the convergence of the PIA. Our approach builds on probabilistic representation formulae for solutions of PDEs and their derivatives. Moreover, in the finite horizon model and in the infinite horizon model with large discount factor, the similar arguments lead to a super-exponential rate of convergence without tear. Finally, with some extra efforts we show that our approach can be extended to the diffusion control case in the one dimensional setting, also with a super-exponential rate of convergence.

math.OC↗