An Approximate Solution Method for Large Risk-Averse Markov Decision Processes

Marek Petrik; Dharmashankar Subramanian

Home/Archive/arXiv

arXiv · 1210.4901

An Approximate Solution Method for Large Risk-Averse Markov Decision Processes

Marek Petrik·Dharmashankar Subramanian

View original source ↗Read source document ↗Export metadata ↓

Abstract

Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse MDPs with hybrid continuous-discrete state spaces and continuous action spaces. The proposed method iteratively improves a bound on the value function using a linearity structure of the MDP. We demonstrate the utility and properties of the method on a portfolio optimization problem.

Explore related subjects

q-fin.PM cs.AI cs.GT

Keep this discovery

BibTeX RIS

Marek Petrik, Dharmashankar Subramanian. 2012-10-16. An Approximate Solution Method for Large Risk-Averse Markov Decision Processes. https://arxiv.org/abs/1210.4901

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Entropic Value-at-Risk parity for tempered stable returns

We develop Entropic Value-at-Risk (EVaR) parity for tempered stable returns. EVaR-based inverse risk parity (IRP) and equal risk contribution (ERC) portfolios are constructed using multivariate normal tempered stable models and independent component analysis with tempered stable components. We derive the corresponding asset-level EVaR and EVaR-deviation contributions and use the latter to separate the fitted location term from EVaR risk contributions. Under Gaussian returns, EVaR-deviation IRP and ERC recover conventional volatility IRP and ERC weights. We evaluate the resulting portfolios in three investment universes. Empirically, EVaR-based ERC portfolios achieve positive Sharpe differences relative to equal weight across the universes.

q-fin.PM↗

arXiv2026

Simple Dynamic Stock/Bond/Gold Portfolios

For more than four decades, the 60/40 stock/bond portfolio has served as a benchmark for delivering reasonable returns without excessive risk. More recently, a 50/30/20 stock/bond/alternative portfolio has been suggested. We use gold as the alternative and as an inflation hedge. In this paper we ask: how much improvement over these benchmark fixed-weight portfolios can be obtained using widely available public data and standard methods from quantitative finance? We restrict ourselves to long-only dynamic portfolios of stocks, bonds, and gold, plus cash, rebalancing monthly, using only publicly available data. We evaluate portfolios on the conventional metrics: return, volatility, Sharpe ratio (computed in excess of the federal funds rate), drawdown, and turnover, in addition to consistency of performance over time, judged by the consistency of the realized annual volatility. Over the 20--year period 2006--2026, using a conservative estimate of trading costs, we show that all risk-adjusted and drawdown metrics are improved using simple volatility control, where we dynamically mix the fixed-weight portfolios with cash so as to target a fixed volatility. This method relies on a simple estimate of portfolio volatility derived from past returns. We also demonstrate that more sophisticated portfolios based on convex optimization---similar to those used in quantitative hedge funds---yield further substantial improvement in return and risk-adjusted return. We consider two such portfolios, one that uses a simple estimate of future returns based on past returns, and one that forecasts future returns based on past returns and just a handful of widely available public economic data. These portfolios also outperform a suite of standard risk-based allocation methods, such as risk parity and minimum variance, evaluated on the same assets and data.

q-fin.PM↗

arXiv2026

An Entropic Factor Model for Robust Portfolio Replication

Portfolio replication, or the construction of a tradable basket of assets to match the risk-return profile of a target benchmark, is fundamentally an ill-posed inverse problem. When restricted to a subset of available assets, classical variance-minimizing models often yield unstable, over-leveraged portfolios highly vulnerable to market shocks. We propose a unified, two-stage methodology rooted in information theory to achieve robust portfolio replication. First, we model the constituent asset returns against target factors, estimating parameters within data-driven empirical bounds via an entropy minimization principle. Second, using the same entropic approach, we determine the optimal weight replication. In both cases we use an entropy function of the Fermi-Dirac type defined directly on sets of constraints of the inverse problem. We validate this Entropic Factor Model (EFM) against standard Ordinary Least Squares (OLS) across five numerical experiments, including standard equity tracking, multi-asset synthesis, and severe stress-test scenarios. Empirical results demonstrate that the EFM consistently outperforms OLS in terms of annualized turnover and net-of-fees returns. Crucially, during the COVID-19 market crash and under severe idiosyncratic data corruption, the entropic framework acts as a probabilistic ``circuit breaker", defensively reducing capital allocation to compromised assets and providing a highly robust, risk-averse solution for generalized portfolio replication.

q-fin.PM↗