Exponential Adaptive Smoothing and Importance Sampling for Optimization of the Conditional Value-at-Risk

Abstract

We present a novel method for solving conditional value-at-risk (CVaR) optimization problems based on the dual representation of CVaR, which is defined as the worst-case expectation over a risk envelope. The method is based on the Bregman proximal point algorithm and alternates between stochastic primal and dual stages. Every (inner) primal stage involves a subproblem solved by sampling from a probability distribution updated at each dual stage (outer iteration). The likelihood ratio of the dual probability distributions relative to the distribution underlying the original problem converges to the risk identifier of the solution's CVaR. Thus, the dual distribution provides the algorithm with a built-in importance sampling mechanism that draws from the tail of the underlying distribution. Because only samples in the tail influence the CVaR, and samples outside the tail are drawn with decreasing probability, the algorithm delivers exceptional performance over other stochastic approximation methods. We prove the convergence of the algorithm for convex objective functions. Our numerical experiments target representative problems in financial mathematics and machine learning, focusing on portfolio optimization and support-vector machines, respectively.