arXiv subjects

Harry Zheng

Publications and source records attributed to Harry Zheng.

At least 19 recordsLinked to original sources

arXiv2026

Indefinite Stochastic Linear-Quadratic Optimal Control Problem for a Markov Regime-Switching Model

This paper investigates an indefinite stochastic linear-quadratic (SLQ) control problem with parameters subject to Markov regime-switching. Based on the well-posedness of the SLQ problem, we introduce a relaxed compensator that extends SLQ control problems from the positive definite case to the indefinite case. We analyze the corresponding stochastic Hamiltonian system for both unconstrained and constrained control cases under the indefinite framework and derive the corresponding optimal open-loop controls. We further investigate the associated Riccati equations for both unconstrained and constrained control cases and derive the closed-loop feedback forms of optimal controls. We illustrate the theoretical results with an equity-bond asset allocation problem under un-constrained and non-negative control constraints. Numerical simulations validate the effectiveness of the theoretical framework and demonstrate its practical value in solving complex stochastic control problems with Markov regime-switching.

math.OC↗

arXiv2026

Reinforcement Learning for Speculative Trading under Exploratory Framework

We study a speculative trading problem within the exploratory reinforcement learning (RL) framework of Wang et al. [2020]. The problem is formulated as a sequential optimal stopping problem over entry and exit times under general utility function and price process. We first consider a relaxed version of the problem in which the stopping times are modeled by the jump times of Cox processes driven by bounded, non-randomized intensity controls. Under the exploratory formulation, the agent's randomized control is characterized via the probability measure over the jump intensities, and their objective function is regularized by Shannon's differential entropy. This yields a system of the exploratory HJB equations and Gibbs distributions in closed-form as the optimal policy. Error estimates and convergence of the RL objective to the value function of the original problem are established. Finally, an RL algorithm is designed, and its implementation is showcased in a pairs-trading application.

q-fin.MF↗

arXiv2026

Linear-quadratic mixed Stackelberg-zero-sum game for mean-field regime switching system

Motivated by a green finance problem, a linear-quadratic Stackelberg differential game for a regime switching system involving one leader and two followers is studied. The two followers engage in a zero-sum differential game, and both the state system and the cost functional incorporate a conditional mean-field term. Applying continuation method and induction method, we first establish the existence and uniqueness of the conditional mean-field forward-backward stochastic differential equations with Markovian switching. Based on it, we prove the unique solvability of Hamiltonian systems associated with the two followers and the leader. Moreover, utilizing stochastic maximum principle, decoupling approach and optimal filtering technique, we obtain the optimal feedback strategies for the two followers and the leader. Employing the theoretical results, we solve the green finance problem with some numerical simulations.

math.OC↗

arXiv2025

Extended HJB Equation for Mean-Variance Stopping Problem: Vanishing Regularization Method

This paper studies the time-inconsistent MV optimal stopping problem via a game-theoretic approach to find equilibrium strategies. To overcome the mathematical intractability of direct equilibrium analysis, we propose a vanishing regularization method: first, we introduce an entropy-based regularization term to the MV objective, modeling mixed-strategy stopping times using the intensity of a Cox process. For this regularized problem, we derive a coupled extended Hamilton-Jacobi-Bellman (HJB) equation system, prove a verification theorem linking its solutions to equilibrium intensities, and establish the existence of classical solutions for small time horizons via a contraction mapping argument. By letting the regularization term tend to zero, we formally recover a system of parabolic variational inequalities that characterizes equilibrium stopping times for the original MV problem. This system includes an additional key quadratic term--a distinction from classical optimal stopping, where stopping conditions depend only on comparing the value function to the instantaneous reward.

math.OC↗

arXiv2025

Neural Network Convergence for Variational Inequalities

We propose an approach to applying neural networks on linear parabolic variational inequalities. We use loss functions that directly incorporate the variational inequality on the whole domain to bypass the need to determine the stopping region in advance and prove the existence of neural networks whose losses converge to zero. We also prove the functional convergence in the Sobolev space. We then apply our approach to solving an optimal investment and stopping problem in finance. By leveraging duality, we convert the nonlinear HJB-type variational inequality of the primal problem into a linear variational inequality of the dual problem and prove the convergence of the primal value function from the dual neural network solution, an outcome made possible by our Sobolev norm analysis. We illustrate the versatility and accuracy of our method with numerical examples for both power and non-HARA utilities as well as high-dimensional American put option pricing. Our results underscore the potential of neural networks for solving variational inequalities in optimal stopping and control problems.

q-fin.MF↗

arXiv2025

S-shaped Utility Maximization with VaR Constraint and Partial Information

We study S-shaped utility maximisation with VaR constraint and unobservable drift coefficient. Using the Bayesian filter, the concavification principle, and the change of measure, we give a semi-closed integral representation for the dual value function and find a critical wealth level that determines if the constrained problem admits a unique optimal solution and Lagrange multiplier or is infeasible. We also propose three algorithms (Lagrange, simulation, deep neural network) to solve the problem and compare their performances with numerical examples.

q-fin.MF↗

arXiv2025

Fractional-Boundary-Regularized Deep Galerkin Method for Variational Inequalities in Mixed Optimal Stopping and Control

Mixed optimal stopping and stochastic control problems define variational inequalities with non-linear Hamilton-Jacobi-Bellman (HJB) operators, whose numerical solution is notoriously difficult and lack of reliable benchmarks. We first use the dual approach to transform it into a linear operator, and then introduce a Fractional-Boundary-Regularized Deep Galerkin Method (FBR-DGM) that augments the classical $L^2$ loss with Sobolev-Slobodeckij norms on the parabolic boundary, enforcing regularity and yielding consistent improvements in the network approximation and its derivatives. The improved accuracy allows the network to be converted back to the original solution using the dual transform. The self-consistency and stability of the network can be tested by checking the primal-dual relationship among optimal value, optimal wealth, and optimal control, offering innovative benchmarks in the absence of analytical solutions.

math.OC↗

arXiv2025

Convergence of Proximal Policy Gradient Method for Problems with Control Dependent Diffusion Coefficients

We prove convergence of the proximal policy gradient method for a class of constrained stochastic control problems with control in both the drift and diffusion of the state process. The problem requires either the running or terminal cost to be strongly convex, but other terms may be non-convex. The inclusion of control-dependent diffusion introduces additional complexity in regularity analysis of the associated backward stochastic differential equation. We provide sufficient conditions under which the control iterates converge linearly to the optimal control, by deriving representations and estimates of solutions to the adjoint backward stochastic differential equations. We introduce numerical algorithms that implement this method using deep learning and ordinary differential equation based techniques. These approaches enable high accuracy and scalability for stochastic control problems in higher dimensions. We provide numerical examples to demonstrate the accuracy and validate the theoretical convergence guarantees of the algorithms.

math.OC↗

arXiv2025

Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs

Large language models (LLMs) excel at mathematical reasoning and logical problem-solving. The current popular training paradigms primarily use supervised fine-tuning (SFT) and reinforcement learning (RL) to enhance the models' reasoning abilities. However, when using SFT or RL alone, there are respective challenges: SFT may suffer from overfitting, while RL is prone to mode collapse. The state-of-the-art methods have proposed hybrid training schemes. However, static switching faces challenges such as poor generalization across different tasks and high dependence on data quality. In response to these challenges, inspired by the curriculum learning-quiz mechanism in human reasoning cultivation, We propose SASR, a step-wise adaptive hybrid training framework that theoretically unifies SFT and RL and dynamically balances the two throughout optimization. SASR uses SFT for initial warm-up to establish basic reasoning skills, and then uses an adaptive dynamic adjustment algorithm based on gradient norm and divergence relative to the original distribution to seamlessly integrate SFT with the online RL method GRPO. By monitoring the training status of LLMs and adjusting the training process in sequence, SASR ensures a smooth transition between training schemes, maintaining core reasoning abilities while exploring different paths. Experimental results demonstrate that SASR outperforms SFT, RL, and static hybrid training methods.

cs.LG↗

arXiv2024

Mean field games of major-minor agents with recursive functionals

This paper investigates a novel class of mean field games involving a major agent and numerous minor agents, where the agents' functionals are recursive with nonlinear backward stochastic differential equation (BSDE) representations. We term these games "recursive major-minor" (RMM) problems. Our RMM modeling is quite general, as it employs empirical (state, control) averages to define the weak couplings in both the functionals and dynamics of all agents, regardless of their status as major or minor. We construct an auxiliary limiting problem of the RMM by a novel unified structural scheme combining a bilateral perturbation with a mixed hierarchical recomposition. This scheme has its own merits as it can be applied to analyze more complex coupling structures than those in the current RMM. Subsequently, we derive the corresponding consistency condition and explore asymptotic RMM equilibria. Additionally, we examine the RMM problem in specific linear-quadratic settings for illustrative purposes.

math.OC↗

arXiv2024

Deep Learning Methods for S Shaped Utility Maximisation with a Random Reference Point

We consider the portfolio optimisation problem where the terminal function is an S-shaped utility applied at the difference between the wealth and a random benchmark process. We develop several numerical methods for solving the problem using deep learning and duality methods. We use deep learning methods to solve the associated Hamilton-Jacobi-Bellman equation for both the primal and dual problems, and the adjoint equation arising from the stochastic maximum principle. We compare the solution of this non-concave problem to that of concavified utility, a random function depending on the benchmark, in both complete and incomplete markets. We give some numerical results for power and log utilities to show the accuracy of the suggested algorithms.

q-fin.CP↗

arXiv2024

Multistage Robust Average Randomized Spectral Risk Optimization

In this paper, we revisit the multistage spectral risk minimization models proposed by Philpott et al.~\cite{PdF13} and Guigues and R\"omisch \cite{GuR12} but with some new focuses. We consider a situation where the decision maker's (DM's) risk preferences may be state-dependent or even inconsistent at some states, and consequently there is not a single deterministic spectral risk measure (SRM) which can be used to represent the DM's preferences at each stage. We adopt the recently introduced average randomized SRM (ARSRM) (in \cite{li2022randomization}) to describe the DM's overall risk preference at each stage. To solve the resulting multistage ARSRM (MARSRM) problem, we apply the well-known stochastic dual dynamic programming (SDDP) method which generates a sequence of lower and upper bounds in an iterative manner. Under some moderate conditions, we prove that the optimal solution can be found in a finite number of iterations. The MARSRM model generalizes the one-stage ARSRM and simplifies the existing multistage state-dependent preference robust model \cite{liu2021multistage}, while also encompassing the mainstream multistage risk-neutral and risk-averse optimization models \cite{GuR12,PdF13}. In the absence of complete information on the probability distribution of the DM's random preferences, we propose to use distributionally robust ARSRM (DR-ARSRM) to describe the DM's preferences at each stage. We detail computational schemes for solving both MARSRM and DR-MARSRM. Finally, we examine the performance of MARSRM and DR-MARSRM by applying them to an asset allocation problem with transaction costs and compare them with standard risk neutral and risk averse multistage linear stochastic programming (MLSP) models.

math.OC↗

arXiv2024

Consumption-investment optimization with Epstein-Zin utility in unbounded non-Markovian markets

The paper investigates the consumption-investment problem for an investor with Epstein-Zin utility in an incomplete market. A non-Markovian environment with unbounded parameters is considered, which is more realistic in practical financial scenarios compared to the Markovian setting. The optimal consumption and investment strategies are derived using the martingale optimal principle and quadratic backward stochastic differential equations (BSDEs) whose solutions admit some exponential moment. This integrability property plays a crucial role in establishing a key martingale argument. In addition, the paper also examines the associated dual problem and several models within the specified parameter framework.

q-fin.MF↗

arXiv2023

Mean Field Analysis of Two-Party Governance: Competition versus Cooperation among Leaders

This article studies linear-quadratic Stackelberg games between two dominating players (or equivalently, leaders) and a large group of followers, each of whom interacts under a mean field game (MFG) framework. Unlike the conventional major-minor player game, the mean field term herein is endogenously affected by the two leaders simultaneously. These homogeneous followers are non-cooperative, whereas the two leaders can either compete or cooperate with each other, which are respectively formulated as a Nash and a Pareto game. The complete solutions of the leader-follower game can be expressed in terms of the solutions of some non-symmetric Riccati equations. Notably, our analysis suggests that both modes of interactions between leaders has their own merits and neither of them is always more favourable to the community of followers. In our knowledge, a comparative study of the effect of different modes of governance on the society is relatively rare in the existing literature, we here provide its first preliminary quantitative analysis; under a broad class of practically relevant models, we provide sufficient conditions to decide whether cooperation or competition between leaders is more favourable to the followers. Being in common with modern folklore, the relative merits of the two Stackelberg games depend on whether the interests between the two leaders and the followers align among themselves. Representative numerical examples are also supplemented.

math.OC↗

arXiv2022

Speculative Trading, Prospect Theory and Transaction Costs

A speculative agent with Prospect Theory preference chooses the optimal time to purchase and then to sell an indivisible risky asset to maximize the expected utility of the round-trip profit net of transaction costs. The optimization problem is formulated as a sequential optimal stopping problem and we provide a complete characterization of the solution. Depending on the preference and market parameters, the optimal strategy can be "buy and hold", "buy low sell high", "buy high sell higher" or "no trading". Behavioral preference and market friction interact in a subtle way which yields surprising implications on the agent's trading patterns. For example, increasing the market entry fee does not necessarily curb speculative trading, but instead it may induce a higher reference point under which the agent becomes more risk-seeking and in turn is more likely to trade.

q-fin.MF↗

arXiv2022

Optimal Investment, Heterogeneous Consumption and Best Time for Retirement

This paper studies an optimal investment and consumption problem with heterogeneous consumption of basic and luxury goods, together with the choice of time for retirement. The utility for luxury goods is not necessarily a concave function. The optimal heterogeneous consumption strategies for a class of non-homothetic utility maximizer are shown to consume only basic goods when the wealth is small, to consume basic goods and make savings when the wealth is intermediate, and to consume almost all in luxury goods when the wealth is large. The optimal retirement policy is shown to be both universal, in the sense that all individuals should retire at the same level of marginal utility that is determined only by income, labor cost, discount factor as well as market parameters, and not universal, in the sense that all individuals can achieve the same marginal utility with different utility and wealth. It is also shown that individuals prefer to retire as time goes by if the marginal labor cost increases faster than that of income. The main tools used in analyzing the problem are from PDE and stochastic control theory including variational inequality and dual transformation. We finally conduct the simulation analysis for the featured model parameters to investigate practical and economic implications by providing their figures.

q-fin.PM↗

arXiv2021

Stochastic Maximum Principle for Optimal Liquidation with Control-dependent Terminal Time

In this paper we study a general optimal liquidation problem with a control-dependent stopping time which is the first time the stock holding becomes zero or a fixed terminal time, whichever comes first. We prove a stochastic maximum principle (SMP) which is markedly different in its Hamiltonian condition from that of the standard SMP with fixed terminal time. We present a simple example in which the optimal solution satisfies the SMP in this paper but fails the standard SMP in the literature.

math.OC↗

arXiv2021

Duality for optimal consumption with randomly terminating income

We establish a rigorous duality theory, under No Unbounded Profit with Bounded Risk, for an infinite horizon problem of optimal consumption in the presence of an income stream that can terminate randomly at an exponentially distributed time, independent of the asset prices. We thus close a duality gap encountered by Vellekoop and Davis in a version of this problem in a Black-Scholes market. Many of the classical tenets of duality theory hold, with the notable exception that marginal utility at zero initial wealth is finite. We use as dual variables a class of supermartingale deflators such that deflated wealth plus cumulative deflated consumption in excess of income is a supermartingale. We show that the space of discounted local martingale deflators is dense in our dual domain, so that the dual problem can also be expressed as an infimum over the discounted local martingale deflators. We characterise the optimal wealth process, showing that optimal deflated wealth is a potential decaying to zero, while deflated wealth plus cumulative deflated consumption over income is a uniformly integrable martingale at the optimum. We apply the analysis to the Vellekoop and Davis example and give a numerical solution.

q-fin.MF↗