arXiv subjects

Jian-Qiang Hu

Publications and source records attributed to Jian-Qiang Hu.

8 recordsLinked to original sources

arXiv2026

Optimal Data Acquisition for Reinforcement Learning: A Large Deviations Perspective

Data acquisition efficiency is a central challenge in deploying reinforcement learning in business and healthcare operations, where interactions are costly, slow, and often involve humans in the loop. This paper develops a unified large deviations framework for data acquisition in infinite-horizon reinforcement learning. We introduce the exponential decay rate of the policy-selection error probability as a principled efficiency metric and derive a variational characterization of this rate via large deviations theory for Markov chains, yielding a nested optimization problem. Based on this characterization, we formalize two complementary notions of optimality in terms of the optimal solution of the nested problem. Because the resulting program is implicit and generally intractable, we propose a tractable convex relaxation with explicit constraints. We then develop a lazy one-step projected subgradient method to solve the relaxed problem and use its iterates to construct an adaptive data acquisition policy. We prove that the resulting reinforcement learning algorithm is near-robustly optimal under our optimality criterion, up to a constant factor. Finally, we extend the framework to linear function approximation to improve scalability, and numerical experiments support the effectiveness of the proposed approach.

cs.LG↗

arXiv2026

Nonparametric Learning and Earning with One-Point Feedback under Nonstationarity

Firms increasingly rely on dynamic pricing to respond to evolving customer demand, yet in many applications they observe only the revenue generated by a single posted price in each period. At the same time, market conditions may shift gradually or abruptly due to changes in customer preferences, competition, or external shocks. These features create two intertwined challenges: learning the revenue--demand relationship from limited feedback and adapting pricing decisions to a changing environment. We study how a seller can learn and earn effectively under these constraints, without assuming a specific parametric form for demand. We develop a learning framework that updates prices using revenue-based gradient approximations constructed from one observation per period. To address environmental changes, we incorporate a restarting mechanism that periodically refreshes the learning process so that outdated information is discounted. When the degree of nonstationarity is unknown, we further introduce a meta-learning layer to adaptively hedge across multiple restarting schedules. We provide performance guarantees for our approach, showing how cumulative revenue loss relative to a fully informed benchmark depends on both the time horizon and the magnitude of market variation. Simulation experiments using synthetic and real-world data illustrate the effectiveness of the proposed procedures.

cs.LG↗

arXiv2026

Long-Run Conditional Value-at-Risk Reinforcement Learning

Conditional value-at-risk (CVaR) is a prominent risk measure in financial engineering, energy systems, and supply chain management. In these domains, Markov decision processes (MDPs) with a long-run CVaR criterion effectively mitigate cost variability over a specified horizon. However, implementing MDPs relies on known transition models, which are typically unavailable in practice. This necessitates a model-free approach to risk-sensitive dynamic optimization. To tackle this challenge, we propose a reinforcement learning algorithm that simultaneously conducts policy evaluation and improvement based on a CVaR-specific Bellman local optimality equation. This algorithm employs a nonparametric incremental learning approach for policy improvement, relying on a single sample trajectory to identify the optimal policy. Under appropriate technical conditions, we prove almost sure convergence of the algorithm and derive its convergence rate. Our analysis reveals that the optimal convergence rate, measured by the mean absolute error of policy estimators, is of order O(1/n). Our main algorithm and results are further extended to solving the mean-CVaR optimization problem. Numerical experiments corroborate these results.

math.OC↗

arXiv2025

A kernel-based stochastic approximation framework for contextual optimization

We present a kernel-based stochastic approximation (KBSA) framework for solving contextual stochastic optimization problems with differentiable objective functions. The framework only relies on system output estimates and can be applied to address a large class of contextual measures, including conditional expectations, conditional quantiles, CoVaR, and conditional expected shortfalls.Under appropriate conditions, we show the strong convergence of KBSA and characterize its finite-time performance in terms of bounds on the mean squared errors of the sequences of iterates produced. In addition, we discuss variants of the framework, including a version based on high-order kernels for further enhancing the convergence rate of the method and an extension of KBSA for handling contextual measures involving multiple conditioning events.Simulation experiments are also carried out to illustrate the framework.

math.OC↗

arXiv2025

Density Approximation of Affine Jump Diffusions via Closed-Form Moment Matching

We develop a recursive approach for deriving closed-form solutions to both conditional and unconditional moments of affine jump diffusions with state-independent jump intensities. Using these moment solutions, we construct closed-form density approximations (up to a normalization constant) via moment matching for both conditional and unconditional distributions. Our framework enables important financial applications, including efficient option pricing and exact simulation for affine jump diffusions. Numerical experiments demonstrate the method's superior computational efficiency compared to existing simulation techniques, while preserving numerical precision.

q-fin.MF↗

arXiv2025

ajdmom: A Python Package for Deriving Moment Formulas of Affine Jump Diffusion Processes

We introduce ajdmom, a Python package designed for automatically deriving moment formulae for the well-established affine jump diffusion processes with state-independent jump intensities. ajdmom can produce explicit closed-form expressions for conditional and unconditional moments of any order, significantly enhancing the usability of these models. Additionally, ajdmom can compute partial derivatives of these moments with respect to the model parameters, offering a valuable tool for sensitivity analysis. The package's modular architecture makes it easy for adaptation and extension by researchers. ajdmom is open-source and readily available for installation from GitHub or the Python package index (PyPI).

q-fin.MF↗

arXiv2025

Black-box Optimization with Simultaneous Statistical Inference for Optimal Performance

Black-box optimization is often encountered for decision-making in complex systems management, where the knowledge of system is limited. Under these circumstances, it is essential to balance the utilization of new information with computational efficiency. In practice, decision-makers often face the dual tasks of optimization and statistical inference for the optimal performance, in order to achieve it with a high reliability. Our goal is to address the dual tasks in an online fashion. Wu et al (2022) [arXiv preprint: 2210.06737] point out that the sample average of performance estimates generated by the optimization algorithm needs not to admit a central limit theorem. We propose an algorithm that not only tackles this issue, but also provides an online consistent estimator for the variance of the performance. Furthermore, we characterize the convergence rate of the coverage probabilities of the asymptotic confidence intervals.

stat.CO↗

arXiv2024

Method of Moments Estimation for Affine Stochastic Volatility Models

We develop moment estimators for the parameters of affine stochastic volatility models. We first address the challenge of calculating moments for the models by introducing a recursive equation for deriving closed-form expressions for moments of any order. Consequently, we propose our moment estimators. We then establish a central limit theorem for our estimators and derive the explicit formulas for the asymptotic covariance matrix. Finally, we provide numerical results to validate our method.

q-fin.ST↗