Teaching decision theory proof strategies using a crowdsourcing problem

Abstract

Teaching how to derive minimax decision rules can be challenging because of the lack of examples that are simple enough to be used in the classroom. Motivated by this challenge, we provide a new example that illustrates the use of standard techniques in the derivation of optimal decision rules under the Bayes and minimax approaches. We discuss how to predict the value of an unknown quantity, $\theta \! \in \! \{0,1\}$, given the opinions of $n$ experts. An important example of such crowdsourcing problem occurs in modern cosmology, where $\theta$ indicates whether a given galaxy is merging or not, and $Y_1, \ldots, Y_n$ are the opinions from $n$ astronomers regarding $\theta$. We use the obtained prediction rules to discuss advantages and disadvantages of the Bayes and minimax approaches to decision theory. The material presented here is intended to be taught to first-year graduate students.