arXiv subjects

Frédéric Godin

Publications and source records attributed to Frédéric Godin.

18 recordsLinked to original sources

arXiv2026

Insights on Time-consistent Deep Hedging under Elicitable Dynamic Risk Measures

We study deep hedging where the objective is expressed in terms of a dynamic risk measure, under which sequential decisions are time-consistent. We tackle the problem of basket option hedging. We show that the approach is feasible and can be used conveniently in the presence of more complex state spaces with higher dimension than what was previously test in the literature. We rely on the conditional elicitability of spectral risk measures to represent the optimization objective. We provide insights on how the choice of scoring function impacts the training of the hedging agent. We also propose an adapted reinforcement learning (RL) algorithm to solve the problem, which contains additions that are crucial to a successful training of the deep RL agent, and without which the agent fails to converge to proper solutions. The time-consistent hedging strategies are also benchmarked against deep hedging approaches relying on static risk measures leading to precommitment.

q-fin.CP↗

arXiv2026

Is Deep Hedging Reinforcement Learning?

The deep hedging framework of Buehler et al. (2019) trains a neural network policy, via Monte Carlo simulation of price paths and stochastic gradient descent, to minimize a risk measure applied to the terminal hedging error. In a recent stream of papers, my coauthors and I have described this technique as reinforcement learning (RL). Several peers have, on occasion, expressed the view that deep hedging does not constitute genuine RL, on two grounds, among others: first, that because feedback is generated only at the terminal date, with no intermediate reward signal, the method cannot constitute genuine RL; and second, that the absence of a value function, a Bellman equation, temporal-difference (TD) learning, and an explicit exploration mechanism disqualifies the method from the RL category altogether, so that it should instead be labeled a neural-network method for stochastic optimal control. The present note argues instead that both objections rest on an unduly narrow, TD-centric reading of what constitutes RL, and that once RL is understood, as it is in the standard references of the field, to include Monte Carlo policy-gradient methods and direct (actor-only) policy search as first-class members, the deep hedging algorithm of Buehler et al. (2019) falls squarely within the RL umbrella.

q-fin.CP↗

arXiv2025

Learning to Hedge Swaptions

This paper investigates the deep hedging framework, based on reinforcement learning (RL), for the dynamic hedging of swaptions, contrasting its performance with traditional sensitivity-based rho-hedging. We design agents under three distinct objective functions (mean squared error, downside risk, and Conditional Value-at-Risk) to capture alternative risk preferences and evaluate how these objectives shape hedging styles. Relying on a three-factor arbitrage-free dynamic Nelson-Siegel model for our simulation experiments, our findings show that near-optimal hedging effectiveness is achieved when using two swaps as hedging instruments. Deep hedging strategies dynamically adapt the hedging portfolio's exposure to risk factors across states of the market. In our experiments, their out-performance over rho-hedging strategies persists even in the presence some of model misspecification. These results highlight RL's potential to deliver more efficient and resilient swaption hedging strategies.

q-fin.RM↗

arXiv2025

Enhancing Deep Hedging of Options with Implied Volatility Surface Feedback Information

We present a dynamic hedging scheme for S&P 500 options, where rebalancing decisions are enhanced by integrating information about the implied volatility surface dynamics. The optimal hedging strategy is obtained through a deep policy gradient-type reinforcement learning algorithm. The favorable inclusion of forward-looking information embedded in the volatility surface allows our procedure to outperform several conventional benchmarks such as practitioner and smiled-implied delta hedging procedures, both in simulation and backtesting experiments. The outperformance is more pronounced in the presence of transaction costs.

q-fin.RM↗

arXiv2025

Deep Hedging with Options Using the Implied Volatility Surface

We propose a deep hedging framework for index option portfolios, grounded in a realistic market simulator that captures the joint dynamics of S&P 500 returns and the full implied volatility surface. Our approach integrates surface-informed decisions with multiple hedging instruments and explicitly accounts for transaction costs. The hedging strategy also considers the variance risk premium embedded in the hedging instruments, enabling more informed and adaptive risk management. Tested on a historical out-of-sample set of straddles from 2020 to 2023, our method consistently outperforms traditional delta-gamma hedging strategies across a range of market conditions.

q-fin.RM↗

arXiv2025

Deep Reinforcement Learning Algorithms for Option Hedging

Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms have been used to find optimal solutions to dynamic hedging problems by framing them as sequential decision-making problems. However, most previous work assesses the performance of only one or two DRL algorithms, making an objective comparison across algorithms difficult. In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a novel application to the task of dynamic hedging. In our experiments, we use the Black-Scholes delta hedge as a baseline and simulate the dataset using a GJR-GARCH(1,1) model. Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi-quadratic penalty. Moreover, MCPG is the only algorithm to outperform the Black-Scholes delta hedge baseline with the allotted computational budget, possibly due to the sparsity of rewards in our environment.

q-fin.CP↗

arXiv2024

Survival Multiarmed Bandits with Bootstrapping Methods

The Multiarmed Bandits (MAB) problem has been extensively studied and has seen many practical applications in a variety of fields. The Survival Multiarmed Bandits (S-MAB) open problem is an extension which constrains an agent to a budget that is directly related to observed rewards. As budget depletion leads to ruin, an agent's objective is to both maximize expected cumulative rewards and minimize the probability of ruin. This paper presents a framework that addresses such a dual goal using an objective function balanced by a ruin aversion component. Action values are estimated through a novel approach which consists of bootstrapping samples from previously observed rewards. In numerical experiments, the policies we present outperform benchmarks from the literature.

cs.LG↗

arXiv2024

Is the difference between deep hedging and delta hedging a statistical arbitrage?

The recent work of Horikawa and Nakagawa (2024) claims that under a complete market admitting statistical arbitrage, the difference between the hedging position provided by deep hedging and that of the replicating portfolio is a statistical arbitrage. This raises concerns as it entails that deep hedging can include a speculative component aimed simply at exploiting the structure of the risk measure guiding the hedging optimisation problem. We test whether such finding remains true in a GARCH-based market model, which is an illustrative case departing from complete market dynamics. We observe that the difference between deep hedging and delta hedging is a speculative overlay if the risk measure considered does not put sufficient relative weight on adverse outcomes. Nevertheless, a suitable choice of risk measure can prevent the deep hedging agent from engaging in speculation.

q-fin.CP↗

arXiv2024

Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients

This paper tackles the problem of mitigating catastrophic risk (which is risk with very low frequency but very high severity) in the context of a sequential decision making process. This problem is particularly challenging due to the scarcity of observations in the far tail of the distribution of cumulative costs (negative rewards). A policy gradient algorithm is developed, that we call POTPG. It is based on approximations of the tail risk derived from extreme value theory. Numerical experiments highlight the out-performance of our method over common benchmarks, relying on the empirical distribution. An application to financial risk management, more precisely to the dynamic hedging of a financial option, is presented.

cs.LG↗

arXiv2024

Deep Hedging with Market Impact

Dynamic hedging is the practice of periodically transacting financial instruments to offset the risk caused by an investment or a liability. Dynamic hedging optimization can be framed as a sequential decision problem; thus, Reinforcement Learning (RL) models were recently proposed to tackle this task. However, existing RL works for hedging do not consider market impact caused by the finite liquidity of traded instruments. Integrating such feature can be crucial to achieve optimal performance when hedging options on stocks with limited liquidity. In this paper, we propose a novel general market impact dynamic hedging model based on Deep Reinforcement Learning (DRL) that considers several realistic features such as convex market impacts, and impact persistence through time. The optimal policy obtained from the DRL model is analysed using several option hedging simulations and compared to commonly used procedures such as delta hedging. Results show our DRL model behaves better in contexts of low liquidity by, among others: 1) learning the extent to which portfolio rebalancing actions should be dampened or delayed to avoid high costs, 2) factoring in the impact of features not considered by conventional approaches, such as previous hedging errors through the portfolio value, and the underlying asset's drift (i.e. the magnitude of its expected return).

q-fin.CP↗

arXiv2023

Linear pretraining in recurrent mixture density networks

We present a method for pretraining a recurrent mixture density network (RMDN). We also propose a slight modification to the architecture of the RMDN-GARCH proposed by Nikolaev et al. [2012]. The pretraining method helps the RMDN avoid bad local minima during training and improves its robustness to the persistent NaN problem, as defined by Guillaumes [2017], which is often encountered with mixture density networks. Such problem consists in frequently obtaining "Not a number" (NaN) values during training. The pretraining method proposed resolves these issues by training the linear nodes in the hidden layer of the RMDN before starting including non-linear node updates. Such an approach improves the performance of the RMDN and ensures it surpasses that of the GARCH model, which is the RMDN's linear counterpart.

cs.LG↗

arXiv2021

Risk averse non-stationary multi-armed bandits

This paper tackles the risk averse multi-armed bandits problem when incurred losses are non-stationary. The conditional value-at-risk (CVaR) is used as the objective function. Two estimation methods are proposed for this objective function in the presence of non-stationary losses, one relying on a weighted empirical distribution of losses and another on the dual representation of the CVaR. Such estimates can then be embedded into classic arm selection methods such as epsilon-greedy policies. Simulation experiments assess the performance of the arm selection algorithms based on the two novel estimation approaches, and such policies are shown to outperform naive benchmarks not taking non-stationarity into account.

cs.LG↗

arXiv2021

Deep equal risk pricing of financial derivatives with non-translation invariant risk measures

The use of non-translation invariant risk measures within the equal risk pricing (ERP) methodology for the valuation of financial derivatives is investigated. The ability to move beyond the class of convex risk measures considered in several prior studies provides more flexibility within the pricing scheme. In particular, suitable choices for the risk measure embedded in the ERP framework such as the semi-mean-square-error (SMSE) are shown herein to alleviate the price inflation phenomenon observed under Tail Value-at-Risk based ERP as documented for instance in Carbonneau and Godin (2021b). The numerical implementation of non-translation invariant ERP is performed through deep reinforcement learning, where a slight modification is applied to the conventional deep hedging training algorithm (see Buehler et al., 2019) so as to enable obtaining a price through a single training run for the two neural networks associated with the respective long and short hedging strategies. The accuracy of the neural network training procedure is shown in simulation experiments not to be materially impacted by such modification of the training algorithm.

q-fin.CP↗

arXiv2021

Bias-Corrected Peaks-Over-Threshold Estimation of the CVaR

The conditional value-at-risk (CVaR) is a useful risk measure in fields such as machine learning, finance, insurance, energy, etc. When measuring very extreme risk, the commonly used CVaR estimation method of sample averaging does not work well due to limited data above the value-at-risk (VaR), the quantile corresponding to the CVaR level. To mitigate this problem, the CVaR can be estimated by extrapolating above a lower threshold than the VaR using a generalized Pareto distribution (GPD), which is often referred to as the peaks-over-threshold (POT) approach. This method often requires a very high threshold to fit well, leading to high variance in estimation, and can induce significant bias if the threshold is chosen too low. In this paper, we derive a new expression for the GPD approximation error of the CVaR, a bias term induced by the choice of threshold, as well as a bias correction method for the estimated GPD parameters. This leads to the derivation of a new estimator for the CVaR that we prove to be asymptotically unbiased. In a practical setting, we show through experiments that our estimator provides a significant performance improvement compared with competing CVaR estimators in finite samples. As a consequence of our bias correction method, it is also shown that a much lower threshold can be selected without introducing significant bias. This allows a larger portion of data to be be used in CVaR estimation compared with the typical POT approach, leading to more stable estimates. As secondary results, a new estimator for a second-order parameter of heavy-tailed distributions is derived, as well as a confidence interval for the CVaR which enables quantifying the level of variability in our estimator.

stat.ME↗

arXiv2021

Deep Equal Risk Pricing of Financial Derivatives with Multiple Hedging Instruments

This paper studies the equal risk pricing (ERP) framework for the valuation of European financial derivatives. This option pricing approach is consistent with global trading strategies by setting the premium as the value such that the residual hedging risk of the long and short positions in the option are equal under optimal hedging. The ERP setup of Marzban et al. (2020) is considered where residual hedging risk is quantified with convex risk measures. The main objective of this paper is to assess through extensive numerical experiments the impact of including options as hedging instruments within the ERP framework. The reinforcement learning procedure developed in Carbonneau and Godin (2020), which relies on the deep hedging algorithm of Buehler et al. (2019b), is applied to numerically solve the global hedging problems by representing trading policies with neural networks. Among other findings, numerical results indicate that in the presence of jump risk, hedging long-term puts with shorter-term options entails a significant decrease of both equal risk prices and market incompleteness as compared to trading only the stock. Monte Carlo experiments demonstrate the potential of ERP as a fair valuation approach providing prices consistent with observable market prices. Analyses exhibit the ability of ERP to span a large interval of prices through the choice of convex risk measures which is close to encompass the variance-optimal premium.

q-fin.CP↗

arXiv2020

Risk-Averse Action Selection Using Extreme Value Theory Estimates of the CVaR

In a wide variety of sequential decision making problems, it can be important to estimate the impact of rare events in order to minimize risk exposure. A popular risk measure is the conditional value-at-risk (CVaR), which is commonly estimated by averaging observations that occur beyond a quantile at a given confidence level. When this confidence level is very high, this estimation method can exhibit high variance due to the limited number of samples above the corresponding quantile. To mitigate this problem, extreme value theory can be used to derive an estimator for the CVaR that uses extrapolation beyond available samples. This estimator requires the selection of a threshold parameter to work well, which is a difficult challenge that has been widely studied in the extreme value theory literature. In this paper, we present an estimation procedure for the CVaR that combines extreme value theory and a recently introduced method of automated threshold selection by \cite{bader2018automated}. Under appropriate conditions, we estimate the tail risk using a generalized Pareto distribution. We compare empirically this estimation procedure with the commonly used method of sample averaging, and show an improvement in performance for some distributions. We finally show how the estimation procedure can be used in reinforcement learning by applying our method to the multi-arm bandit problem where the goal is to avoid catastrophic risk.

stat.ML↗

arXiv2020

Modeling and measuring incurred claims risk liabilities for a multi-line property and casualty insurer

We propose a stochastic model allowing property and casualty insurers with multiple business lines to measure their liabilities for incurred claims risk and calculate associated capital requirements. Our model includes many desirable features which enable reproducing empirical properties of loss ratio dynamics. For instance, our model integrates a double generalized linear model relying on accident semester and development lag effects to represent both the mean and dispersion of loss ratio distributions, an autocorrelation structure between loss ratios of the various development lags, and a hierarchical copula model driving the dependence across the various business lines. The model allows for a joint simulation of loss triangles and the quantification of the overall portfolio risk through risk measures. Consequently, a diversification benefit associated to the economic capital requirements can be measured, in accordance with IFRS 17 standards which allow for the recognition of such benefit. The allocation of capital across business lines based on the Euler allocation principle is then illustrated. The implementation of our model is performed by estimating its parameters based on a car insurance data obtained from the General Insurance Statistical Agency (GISA), and by conducting numerical simulations whose results are then presented.

q-fin.RM↗

arXiv2020

Equal Risk Pricing of Derivatives with Deep Hedging

This article presents a deep reinforcement learning approach to price and hedge financial derivatives. This approach extends the work of Guo and Zhu (2017) who recently introduced the equal risk pricing framework, where the price of a contingent claim is determined by equating the optimally hedged residual risk exposure associated respectively with the long and short positions in the derivative. Modifications to the latter scheme are considered to circumvent theoretical pitfalls associated with the original approach. Derivative prices obtained through this modified approach are shown to be arbitrage-free. The current paper also presents a general and tractable implementation for the equal risk pricing framework inspired by the deep hedging algorithm of Buehler et al. (2019). An $ε$-completeness measure allowing for the quantification of the residual hedging risk associated with a derivative is also proposed. The latter measure generalizes the one presented in Bertsimas et al. (2001) based on the quadratic penalty. Monte Carlo simulations are performed under a large variety of market dynamics to demonstrate the practicability of our approach, to perform benchmarking with respect to traditional methods and to conduct sensitivity analyses.

q-fin.CP↗