SearcharxivSearch

arXiv · 2411.12746

A Review of Reinforcement Learning in Financial Applications

Abstract

In recent years, there has been a growing trend of applying Reinforcement Learning (RL) in financial applications. This approach has shown great potential to solve decision-making tasks in finance. In this survey, we present a comprehensive study of the applications of RL in finance and conduct a series of meta-analyses to investigate the common themes in the literature, such as the factors that most significantly affect RL's performance compared to traditional methods. Moreover, we identify challenges including explainability, Markov Decision Process (MDP) modeling, and robustness that hinder the broader utilization of RL in the financial industry and discuss recent advancements in overcoming these challenges. Finally, we propose future research directions, such as benchmarking, contextual RL, multi-agent RL, and model-based RL to address these challenges and to further enhance the implementation of RL in finance.

Explore related subjects

Keep this discovery

BibTeXRIS

Yahui Bai, Yuhe Gao, Runzhe Wan, Sheng Zhang, Rui Song. 2024-11-01. A Review of Reinforcement Learning in Financial Applications. https://arxiv.org/abs/2411.12746

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Unbiased Monte Carlo Greeks for Discontinuous Payoffs

Pathwise differentiation of Monte Carlo estimators fails at payoff discontinuities, producing zero or biased sensitivities for barriers, autocallables, and digital options. The industry workaround --- smoothing the indicator functions --- introduces bias and requires per-product calibration. We derive a correction formula that restores unbiased Greeks without smoothing. For a payoff $F(Z,\theta)$ that is piecewise smooth with discontinuities on surfaces $\{g_i = 0\}$, we show that the sensitivity decomposes into a pathwise term (computed by standard AAD) plus a sum of boundary corrections, each involving the payoff jump, the Gaussian density at the boundary, and the sensitivity of the boundary to the parameter. The correction is computed by Newton root-finding in the normal-random space, with the jump evaluated by two forward replays of the pricing kernel. The implementation uses AADC (\texttt{pip install aadc}), whose tape replay and automatic discontinuity tracking make the method fully automatic --- the quant writes standard pricing code, and the correction driver identifies and handles all discontinuities. We prove the formula for arbitrary compositions of smooth functions and indicator functions (not just outer products), covering real autocallable payoff structures with recursive alive/dead logic. Benchmarks on QuantLib models (GBM, Heston, Hull-White) show all Greeks within 0.1--4\% of analytic or bump-and-revalue references.

q-fin.CP

Global Multi-Maturity SPX-VIX Calibration Beyond Markovian Stitching

We develop a global framework for joint S&P 500 (SPX)-VIX smile calibration across multiple maturities without the conditional-independence restriction induced by Markovian stitching. Exact local and global feasibility are equivalent: every globally feasible law has a block-preserving SPX-Markovization that leaves each monthly $(S_i,V_i,S_{i+1})$ law unchanged. Nevertheless, stitched laws can form a strict subset of globally feasible path laws because Markovization discards dependence on earlier history beyond the current SPX level. Adjacent smiles therefore cannot identify this dependence, and laws with identical monthly calibrations can price multi-period claims differently. Under the standard Markov reference, relative entropy selects the stitched minimum-information completion; non-Markov dependence requires cross-period information, an appropriate objective, or a history-dependent prior. For finite discretizations, we introduce an augmented-Bregman mirror-descent scheme. It preserves the fit to observable quote moments while controlling martingale and dispersion residuals. In a controlled infeasible affine system, this split keeps prescribed marginals about $25$ times tighter than cyclic row projection by exposing the discrepancy in the conditional rows. An exact finite-state example verifies block preservation and exhibits material cross-period price changes after Markovization. On smoothed SPX and VIX surfaces, numerical calculations illustrate a finite-budget penalty path: the worst fitted-smile error remains below $0.70$ volatility points across the reported sweep while the bulk conditional diagnostics improve substantially.

q-fin.CP

Quantum Circuit Learning for Volatility Modeling: Multifractal Analysis of Realized Volatility Time Series

Herein, we propose a quantum circuit learning framework for modeling the realized volatility (RV) of Bitcoin and investigate the statistical properties of the predicted time series through multifractal analysis. Unlike conventional GARCH-type models, which require a pre-specified functional form for the volatility process, a parameterized quantum circuit directly approximates the volatility function from empirical data, eliminating the need for explicit model selection. Using five-minute Bitcoin price data, we construct daily RV, train a single-qubit parameterized quantum circuit, and generate a long synthetic time series from the optimized quantum circuit. Multifractal Detrended Fluctuation Analysis is applied to calculate the generalized Hurst exponent $h(q)$, the singularity spectrum $f(\alpha)$, and the multifractal scaling exponent $\tau(q)$. The predicted return series exhibits $h(2)\approx 0.5$, consistent with near-random dynamics, and both the predicted and the empirical return series display multifractality that partially persists after random shuffling. The increment series of RV shows pronounced anti-persistence with $h(2)\approx 0.05$--$0.1$, consistent with the rough volatility hypothesis. These results demonstrate that a simple single-qubit parameterized quantum circuit captures qualitatively some observed properties in Bitcoin volatility dynamics.

q-fin.CP