Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading

Koti S. Jaddu; Paul A. Bilokon

Home/Archive/arXiv

arXiv · 2311.02088

Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading

Koti S. Jaddu·Paul A. Bilokon

View original source ↗Read source document ↗Export metadata ↓

Abstract

High-frequency trading is prevalent, where automated decisions must be made quickly to take advantage of price imbalances and patterns in price action that forecast near-future movements. While many algorithms have been explored and tested, analytical methods fail to harness the whole nature of the market environment by focusing on a limited domain. With the evergrowing machine learning field, many large-scale end-to-end studies on raw data have been successfully employed to increase the domain scope for profitable trading but are very difficult to replicate. Combining deep learning on the order books with reinforcement learning is one way of breaking down large-scale end-to-end learning into more manageable and lightweight components for reproducibility, suitable for retail trading. The following work focuses on forecasting returns across multiple horizons using order flow imbalance and training three temporal-difference learning models for five financial instruments to provide trading signals. The instruments used are two foreign exchange pairs (GBPUSD and EURUSD), two indices (DE40 and FTSE100), and one commodity (XAUUSD). The performances of these 15 agents are evaluated through backtesting simulation, and successful models proceed through to forward testing on a retail trading platform. The results prove potential but require further minimal modifications for consistently profitable trading to fully handle retail trading costs, slippage, and spread fluctuation.

Explore related subjects

q-fin.CP cs.AI cs.LG q-fin.PM q-fin.TR

Keep this discovery

BibTeX RIS

Koti S. Jaddu, Paul A. Bilokon. 2023-10-24. Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading. https://arxiv.org/abs/2311.02088

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Unbiased Monte Carlo Greeks for Discontinuous Payoffs

Pathwise differentiation of Monte Carlo estimators fails at payoff discontinuities, producing zero or biased sensitivities for barriers, autocallables, and digital options. The industry workaround --- smoothing the indicator functions --- introduces bias and requires per-product calibration. We derive a correction formula that restores unbiased Greeks without smoothing. For a payoff $F(Z,\theta)$ that is piecewise smooth with discontinuities on surfaces $\{g_i = 0\}$, we show that the sensitivity decomposes into a pathwise term (computed by standard AAD) plus a sum of boundary corrections, each involving the payoff jump, the Gaussian density at the boundary, and the sensitivity of the boundary to the parameter. The correction is computed by Newton root-finding in the normal-random space, with the jump evaluated by two forward replays of the pricing kernel. The implementation uses AADC (\texttt{pip install aadc}), whose tape replay and automatic discontinuity tracking make the method fully automatic --- the quant writes standard pricing code, and the correction driver identifies and handles all discontinuities. We prove the formula for arbitrary compositions of smooth functions and indicator functions (not just outer products), covering real autocallable payoff structures with recursive alive/dead logic. Benchmarks on QuantLib models (GBM, Heston, Hull-White) show all Greeks within 0.1--4\% of analytic or bump-and-revalue references.

q-fin.CP↗

arXiv2026

Global Multi-Maturity SPX-VIX Calibration Beyond Markovian Stitching

We develop a global framework for joint S&P 500 (SPX)-VIX smile calibration across multiple maturities without the conditional-independence restriction induced by Markovian stitching. Exact local and global feasibility are equivalent: every globally feasible law has a block-preserving SPX-Markovization that leaves each monthly $(S_i,V_i,S_{i+1})$ law unchanged. Nevertheless, stitched laws can form a strict subset of globally feasible path laws because Markovization discards dependence on earlier history beyond the current SPX level. Adjacent smiles therefore cannot identify this dependence, and laws with identical monthly calibrations can price multi-period claims differently. Under the standard Markov reference, relative entropy selects the stitched minimum-information completion; non-Markov dependence requires cross-period information, an appropriate objective, or a history-dependent prior. For finite discretizations, we introduce an augmented-Bregman mirror-descent scheme. It preserves the fit to observable quote moments while controlling martingale and dispersion residuals. In a controlled infeasible affine system, this split keeps prescribed marginals about $25$ times tighter than cyclic row projection by exposing the discrepancy in the conditional rows. An exact finite-state example verifies block preservation and exhibits material cross-period price changes after Markovization. On smoothed SPX and VIX surfaces, numerical calculations illustrate a finite-budget penalty path: the worst fitted-smile error remains below $0.70$ volatility points across the reported sweep while the bulk conditional diagnostics improve substantially.

q-fin.CP↗

arXiv2026

Quantum Circuit Learning for Volatility Modeling: Multifractal Analysis of Realized Volatility Time Series

Herein, we propose a quantum circuit learning framework for modeling the realized volatility (RV) of Bitcoin and investigate the statistical properties of the predicted time series through multifractal analysis. Unlike conventional GARCH-type models, which require a pre-specified functional form for the volatility process, a parameterized quantum circuit directly approximates the volatility function from empirical data, eliminating the need for explicit model selection. Using five-minute Bitcoin price data, we construct daily RV, train a single-qubit parameterized quantum circuit, and generate a long synthetic time series from the optimized quantum circuit. Multifractal Detrended Fluctuation Analysis is applied to calculate the generalized Hurst exponent $h(q)$, the singularity spectrum $f(\alpha)$, and the multifractal scaling exponent $\tau(q)$. The predicted return series exhibits $h(2)\approx 0.5$, consistent with near-random dynamics, and both the predicted and the empirical return series display multifractality that partially persists after random shuffling. The increment series of RV shows pronounced anti-persistence with $h(2)\approx 0.05$--$0.1$, consistent with the rough volatility hypothesis. These results demonstrate that a simple single-qubit parameterized quantum circuit captures qualitatively some observed properties in Bitcoin volatility dynamics.

q-fin.CP↗