Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability

Abstract

We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty. Traditional regret-minimisation bandit strategies like Thompson Sampling and Upper Confidence Bound (UCB) typically rely on distributional assumptions or asymptotic guarantees; further, they remain largely focused on regret, neglecting their statistical properties. We address this gap. Through the adoption of CP, we bridge the regret-minimising potential of a decision-making bandit policy with statistical guarantees in the form of finite-sample prediction coverage. We demonstrate the potential of Conformal Bandits through simulation studies and an application to portfolio allocation, a typical scenario where differences in arm rewards are far too small (weak arm separability) for classical policies to be optimal in finite sample. We showcase our framework's practical advantage in terms of regret in this setting, as well as its added value in achieving nominal coverage guarantees where classical UCB policies may fail. Focusing on our application of interest, we further illustrate how integrating hidden Markov models to capture the regime-switching behaviour of financial markets, enhances the exploration-exploitation trade-off, and translates into higher risk-adjusted returns, while preserving coverage guarantees.