arXiv subjects

Xun Li

Publications and source records attributed to Xun Li.

At least 19 recordsLinked to original sources

arXiv2026

Machine learning emulators of dynamic global vegetation models for fast gridded carbon budget predictions

The land carbon sink explains most of the variability of the atmospheric CO2 growth rate, but the Global Carbon Budget estimates it with dynamic global vegetation models (DGVMs) run once a year. Emulators could provide it sooner if they reproduce each model outside its training period. Here we show that convolutional neural network emulators of 19 of these models, driven by ERA5 climate and their own past predictions, reproduce the global land sink of these models and its spread over two years not used in training. Global monthly net biome production (NBP) anomalies were reproduced with a median R^2 of 0.81. The error of the multi-model mean annual NBP was at most 0.23 GtC yr^-1. At the grid-cell level, the median R^2 of monthly anomalies was 0.30, partly because of errors that accumulate in the emulated NBP history.

physics.ao-ph↗

arXiv2026

Optimal portfolio and consumption under no-shorting constraints: The exploratory formulation and reinforcement learning

We develop a reinforcement-learning framework for the continuous-time optimal portfolio-consumption via utility maximization under no-shorting portfolio constraints. We adopt the exploratory formulation under entropy regularization and impose no-shorting constraint through a componentwise nonnegative conditional-mean constraint on the class of admissible randomized policies, while the sampled actions are allowed to be negative during the training. For CRRA utility, we derive the homothetic value function and the optimal Gaussian portfolio and Gibbs consumption policies under some model conditions and establish a verification theorem. These results allow compatible parameterizations in actor, critic, and $q$ functions for a constrained policy-iteration algorithm. Policy evaluation uses trajectory-based moment conditions, and policy improvement uses fitted coefficients to solve a convex quadratic program over the nonnegative orthant without requiring market coefficients as inputs. The mean constraint preserves tractable Gaussian exploration throughout parameter learning, while modal deployment converts the learned randomized policies into deterministic controls. In particular, the Gaussian mode coincides with its constrained mean and hence satisfies no-shorting constraint, and the Gibbs mode determines the deployed consumption rule.

q-fin.PM↗

arXiv2026

Entropy-Regularized Reinforcement Learning for the Regime-Switching Consumption-Investment Problem

This paper establishes an entropy-regularized reinforcement learning (ERRL) framework for the continuous-time consumption-investment problem under a regime-switching market model with general utility functions. By incorporating randomized exploratory policies and Shannon entropy regularization into the stochastic optimal control formulation, we derive closed-form representations for the optimal exploratory consumption and investment strategies. Specifically, the optimal investment strategy follows a multivariate Gaussian distribution, whereas the optimal consumption strategy takes the form of a Gibbs measure---which specializes to a Gamma distribution under logarithmic utility and a non-standard exponential distribution under power utility. Crucially, our theoretical analysis reveals that the classical deterministic optimal decisions correspond to the mode---rather than the mean---of these exploratory distributions. Furthermore, we establish the convergence of the exploratory value function to its classical unregularized counterpart as the temperature parameter vanishes. For algorithmic design, we develop a continuous-time q-learning framework grounded in a novel continuous-time q-function, prove the policy improvement theorem, and establish martingale characterizations for both on-policy and off-policy learning. We find that parameterizing the actor network based on theoretical distribution families can help accelerate convergence and enhance numerical stability. Furthermore, numerical experiments across regime-switching environments provide empirical support for our theoretical findings, demonstrating stable loss convergence as well as the monotonicity and concavity of the learned value functions.

cs.LG↗

arXiv2026

TADP: Dual-Branch Temporally Adaptive Differential Privacy for Expression-Aware Video Anonymization

We present TADP, a temporally adaptive differential privacy (DP) framework for inference-time privacy-preserving video generation under fixed-length clip-replacement adjacency. TADP uses a dual-branch design to extract identity and expression representations, clips and privatizes both branches with temporally adaptive Gaussian mechanisms whose noise scales depend only on preceding privatized releases within the same clip and are bounded below by history-uniform calibrated minima, and accounts for their joint release via adaptive Rényi Differential Privacy (RDP) composition. The clip-level RDP guarantee is converted to $(\varepsilon,δ)$-differential privacy, while soft quantization and subsequent temporal processing are applied only after privatization as DP post-processing. Experiments on facial expression datasets show reduced identity leakage under evaluated attacks, usable perceptual quality and temporal stability, and high facial expression recognition utility relative to the evaluated privacy-preserving baselines. TADP formally satisfies clip-level DP and empirically provides a favorable privacy--utility trade-off under the evaluated threat models and datasets.

cs.CV↗

arXiv2026

Reinforcement learning for constrained stochastic linear--quadratic control in continuous time

We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.

math.OC↗

arXiv2026

Continuous-Time Mean-Variance Portfolio Selection Reinforcement Learning Problems for a Markov Regime-Switching System with No-Shorting Constraints

This paper is the first to study the continuous-time mean-variance portfolio selection problem with no-shorting constraints in a Markov regime-switching market using reinforcement learning (RL). The main challenge stems from the interaction between random regime shifts and portfolio non-negativity constraints, which typically causes the coupled Hamilton-Jacobi-Bellman (HJB) equations to lose smoothness across different state regions. Additionally, directly imposing no-shorting constraints on randomized policies disrupts their analytically tractable Gaussian structure, complicating both exploratory modeling and reinforcement learning. To address these challenges, we propose an auxiliary exploratory approach that limits the policy to a Gaussian distribution and applies the no-shorting constraint solely to its mean. We explicitly characterize the optimal policy for this auxiliary problem and demonstrate that, by appropriately selecting the exploration variance, its optimal mean policy exactly matches the optimal policy of the original constrained mean-variance problem. Building upon this analytical framework, we develop a model-free reinforcement learning algorithm that learns the optimal portfolio policy directly from trajectory data. Numerical experiments support the theoretical results and show that the proposed method can effectively recover the optimal no-shorting portfolio in a regime-switching market.

math.OC↗

arXiv2026

Policy Evaluation and Policy Gradient for Continuous-Time Markov Regime-Switching Systems

We study policy evaluation (PE) and policy gradient (PG) for continuous-time stochastic control with Markov regime-switching. The main challenges are the coexistence of diffusion and regime jumps, the coupling of regime-dependent value functions, and the construction of model-free learning rules from the coupled generators. We establish a martingale characterization of PE that incorporates both Brownian and switching martingales, and extend martingale loss and orthogonality methods to this setting. We further derive a coupled PG equation and a trajectory-based gradient representation that, under an action-independent switching generator, avoids explicit knowledge of the system coefficients and transition rates. These results lead to offline and online actor-critic algorithms. We also establish convergence and study the finite-grid approximation required for implementation. Numerical experiments for a regime-switching linear-quadratic problem illustrate the proposed methods.

math.OC↗

arXiv2026

Optimal Transport for Continuous-Time Mean-Variance Portfolio Selection

The continuous-time mean-variance portfolio selection problem is investigated in this paper through a rigorous integration of backward stochastic differential equation (BSDE) and Kantorovich duality theory. We propose a novel analytical framework for addressing feedback control in McKean-Vlasov stochastic systems. By solving three coupled systems of ordinary differential equations (ODEs), we derive an explicit, closed-loop optimal feedback investment strategy.

math.OC↗

arXiv2026

Entropy Regularization under No-Shorting: Structure Preservation and Convexity in Dynamic Mean-Variance Portfolio Selection

This paper studies entropy regularization for the discrete-time dynamic mean--variance portfolio problem with no-shorting constraints. In the unconstrained setting, entropy regularization can be introduced without changing the basic quadratic structure of the dynamic program. Under no-shorting, this property is lost: a standard fixed-temperature formulation generally preserves convexity but no longer preserves the homogeneous Bellman structure and the associated piecewise-linear feedback form. We introduce a state-dependent temperature that is matched to the homogeneity of the original mean--variance problem. This restores the piecewise-quadratic value function and the finite-dimensional Bellman recursion. The resulting formulation, however, makes convexity depend on the continuation coefficients generated by the soft recursion. We show that negative continuation coefficients lead to loss of convexity and prove that a common sufficiently small temperature preserves the Bellman structure and convexity over the entire finite horizon. The original no-shorting solution is recovered as the temperature vanishes. We also develop a sample-based implementation for unknown return distributions. Numerical and empirical results illustrate both the structural advantage of the state-dependent temperature and the practical relevance of the convexity-preserving temperature restriction.

q-fin.PM↗

arXiv2026

Mean-variance portfolio selection: connections between mean-field systems and forward-backward stochastic systems under initial-terminal cost coupling

This paper investigates the connection between mean-field systems and forward-backward stochastic systems arising from the mean-variance portfolio selection problem. By introducing the conditional expectation process of terminal wealth, the mean-field term is represented by a backward state, leading to a class of control problems for forward-backward stochastic systems with coupled initial-terminal values. The coupling between the terminal value of the forward state and the initial value of the backward state in the cost functional prevents classical methods for forward-backward stochastic optimal control from being directly applicable. To overcome this difficulty, we establish a stochastic maximum principle for control problems with running costs and coupled initial-terminal values, and provide sufficient conditions for optimality. Furthermore, we investigate a class of generalized stochastic linear-quadratic control problems with initial-terminal coupling, allowing the control weight in the running cost to be indefinite, and derive the optimal feedback representation via a Riccati equation. The classical mean-variance portfolio selection model is shown to be a special case, for which the corresponding efficient investment strategy and efficient frontier are obtained explicitly, and numerical simulations are presented to illustrate the results and examine parameter sensitivity and its financial implications.

math.OC↗

arXiv2026

Stochastic linear-quadratic control problem for regime-switching jump-diffusion system and its application to finance

This paper investigates a stochastic linear-quadratic (SLQ) control problem for a regime-switching jump-diffusion system. Unlike traditional regime-switching diffusion systems that couple a diffusion process with a Markov chain, we incorporate the jumps of the Markov chain into the state equation. This modeling methodology effectively captures potential gains or losses of the system during the regime transitions. It should be noted that the introduction of Markov chain jumps into the state equation leads to increased complexity in the corresponding coupled differential Riccati equations (CDREs), thereby rendering the solvability of the control problem more challenging. Under the assumption that the cost functional is uniformly convex, we establish the unique solvability of the corresponding CDREs. Building upon this foundation, we derive a closed-loop representation for the unique open-loop optimal control. Finally, we apply our theoretical results to the mean-variance portfolio selection problem in a regime-switching financial market with Markov chain jumps and obtain its efficient frontier.

math.OC↗

arXiv2026

Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets

We study continuous-time mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets from a reinforcement learning (RL) perspective. The problem is a constrained stochastic linear -- quadratic control problem for which the entropy-regularized exploratory formulation of Wang et al. (2020) leads to difficulty in theoretical analysis, because enforcing the constraint on the support of randomized policies nullifies the tractable Gaussian exploration. To tackle this challenge, we introduce an auxiliary exploratory problem without entropy in which exploratory policies are still Gaussian whose samples may violate the no-shorting requirement but their means satisfy it. We then prove that, for a suitable choice of exploration variance, the mean of the optimal Gaussian policy of the auxiliary problem coincides with the optimal policy of the original problem. Motivated by this theoretical result, we develop a model-free RL algorithm that learns the optimal policy of the auxiliary (and hence the original) problem directly from trajectory data without estimating the investment opportunity set. A numerical example demonstrates the performance of the proposed algorithm.

math.OC↗

arXiv2026

Closed-loop $α$-Potential Stochastic Differential Games via a BSDE Approach

In this paper, we study the closed-loop $α$-potential stochastic differential game (SDG) problem as a continuation of our prior research on open-loop control (see \cite{GLZ2025}). By utilizing the backward stochastic differential equation (BSDE) approach, we derive a precise estimate for the parameter $α$. Compared to our earlier work \cite{GLZ2025}, this study incorporates both first- and second-order sensitivity state processes, as well as the sensitivity of the control process. A distinguishing feature of this work is that, in the context of $N$-player heterogeneous agent games involving mean-field type interactions, we derive an $N$-uniform upper bound for the minimal potential approximation error. In contrast to the corresponding open-loop estimates, the closed-loop bound contains feedback-induced contributions that need not vanish with $N$. Consequently, our present estimate does not in general guarantee $α\rightarrow 0$.

math.OC↗

arXiv2026

Continuous-time q-learning for Markov regime switching system under Tsallis entropy

This paper studies continuous-time q-learning (the continuous-time counterpart of Q-learning) for a Markov regime-switching system under Tsallis entropy regularization. The Tsallis entropy regularization yields an optimal policy distribution that may not necessarily be a Gibbs measure, thereby complicating algorithm design. Furthermore, to address the limited universality of current continuous-time regime-switching reinforcement learning algorithms (often restricted to the exploratory mean-variance framework), this study focuses on continuous-time q-learning for Markov regime-switching systems based on Tsallis entropy, aiming for a more universally applicable continuous-time reinforcement learning method. We establish the martingale characterization of the q-function under Tsallis entropy for continuous-time Markov regime-switching systems. We further design two q-learning algorithms that differ based on whether the Lagrange multiplier can be explicitly derived. We apply these algorithms to the continuous-time exploratory mean-variance portfolio optimization problem in a regime-switching market. Numerical experiments demonstrate the satisfactory performance of our q-learning algorithms.

math.OC↗

arXiv2026

Tree-of-Ideas: Automated Research Ideation via Cross-Trajectory Reasoning over Scholarly Evolution

Effective research ideation requires moving beyond a static understanding of prior work to trace how research problems and solutions evolve across the literature. Existing methods either treat papers as unstructured context or model scholarly evolution as isolated citation chains, overlooking interactions among research trajectories. We propose Tree-of-Ideas (ToI), a two-stage framework. EvoTrace reconstructs branching scholarly trajectories from citations, tracking evolving methods, resolved problems, and gaps. EvoAgent then reasons across trajectories to identify convergent problems and complementary solutions, generating grounded research ideas. Across six AI research topics, ToI achieves the highest score among automatic methods (6.27 vs. 5.36 for the strongest baseline on a 10-point scale), with strong Novelty (6.36) and Groundedness (7.00). Also, its score approaches that of human-paper references (6.29), demonstrating the value of cross-path evolutionary reasoning.

cs.AI↗

arXiv2026

Inverse reinforcement learning for indefinite mean-field social optimization with multiplicative noise

This paper studies the inverse reinforcement learning (RL) problem for linear-quadratic mean-field (MF) social optimization. The considered system features multiplicative noise and indefinite cost weights, which violate standard convexity assumptions and pose analytical challenges. The goal is to recover unknown social cost weights from expert demonstrations and reproduce the optimal control policies. This requires solving coupled stochastic algebraic Riccati equations and Lyapunov equations with unknown system dynamics. To this end, we first propose a model-based inverse RL algorithm with two sequential loops that separately handle individual and MF dynamics, and we prove its convergence and closed-loop stabilizability. Moreover, we characterize the non-uniqueness of the recovered cost weights. To eliminate reliance on system dynamics, we develop a model-free inverse RL algorithm using integral RL and least-squares identification, which requires only measured trajectory data satisfying mild rank conditions. Finally, numerical simulations validate the effectiveness of the proposed approaches.

math.OC↗

arXiv2026

Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching

This paper addresses infinite-horizon continuous-time stochastic linear quadratic optimal control problems with regime switching. We propose a paradigm shift from model-based design by adopting an adaptive dynamic programming approach, specifically developing on-policy and off-policy Q-learning algorithms that learn the optimal controller solely from online state trajectory data. The theoretical core of our work consists of a complete proof of the equivalence between the on- and off-policy architectures, alongside a rigorous analysis establishing the stability of the closed-loop system and the convergence of the algorithms to the optimal solution. For computational tractability, we implement these algorithms using vectorization and Kronecker product algebra. The theoretical results are corroborated by numerical case studies that clearly demonstrate the operational effectiveness and practical feasibility of the proposed model-free control strategy.

math.OC↗

arXiv2026

Constrained Zero-Sum Stochastic Linear-Quadratic Differential Game for Jump-Diffusion Systems with Random Coefficients

This paper studies a two-player zero-sum stochastic linear-quadratic (SLQ) differential game for controlled jump-diffusion systems with random coefficients, where the controls of both players are constrained to nonempty closed convex cones. Under a uniform convexity--concavity condition, we establish the existence and uniqueness of an open-loop saddle point and characterize it by a forward--backward stochastic differential equation with jumps (FBSDEJ) together with cone-type variational inequalities. Assuming the existence of positive bounded solutions to the associated system of indefinite extended stochastic Riccati equations with jumps (IESREJs), we derive a feedback-form representation of the unique open-loop saddle point by constructing predictable minimax selectors and combining the Meyer--Itô formula with jumps, and the FBSDEJ characterization. Finally, under additional structural conditions, we prove the existence of positive bounded solutions to the IESREJs by a double-truncation approximation and a multidimensional BSDEJ comparison theorem.

math.OC↗