Gaussian One-Armed Bandit and Optimization of Batch Data Processing

Abstract

We consider the minimax setup for Gaussian one-armed bandit problem, i.e. the two-armed bandit problem with Gaussian distributions of incomes and known distribution corresponding to the first arm. This setup naturally arises when the optimization of batch data processing is considered and there are two alternative processing methods available with a priori known efficiency of the first method. One should estimate the efficiency of the second method and provide predominant usage of the most efficient of both them. According to the main theorem of the theory of games minimax strategy and minimax risk are searched for as Bayesian ones corresponding to the worst-case prior distribution. As a result, we obtain the recursive integro-difference equation and the second order partial differential equation in the limiting case as the number of batches goes to infinity. This makes it possible to determine minimax risk and minimax strategy by numerical methods. If the number of batches is large enough we show that batch data processing almost does not influence the control performance, i.e. the value of the minimax risk. Moreover, in case of Bernoulli incomes and large number of batches, batch data processing provides almost the same minimax risk as the optimal one-by-one data processing.