Binary sampling from discrete distributions

Abstract

This paper considers direct sampling methods from discrete target distributions. The inverse transform sampling (ITS) method is one of the most popular direct sampling methods. The main purpose of this paper is to propose a direct sampling algorithm that supersedes the binary-search ITS method (which is an improvement of the ITS method with binary search). The proposed algorithm is based on binarizing the support set of the target distribution. Thus, the proposed algorithm is referred to as binary sampling (BS). The BS algorithm consists of two procedures: backward binary sampling (BBS) and forward binary sampling (FBS). The BBS procedure draws a single sample (the first sample) from the target distribution while constructing a one-way random walk on a binary tree for the FBS procedure. By running the random walk, the FBS procedure generates the second and subsequent samples. The BBS and FBS procedures have $O(N)$ and $O(\ln N)$ time complexities, respectively, and they also have $O(N)$ space complexity, where $N+1$ is the cardinality of the support set of the target distribution. Therefore, the time and space complexities of the BS algorithm are equivalent to those of the standard (possibly best) binary-search ITS algorithm. However, the BS algorithm has two advantages over the standard binary-search ITS algorithm. First, the BBS procedure is parallelizable and thus the total running time of the BS algorithm can be reduced. Second, the BS algorithm is more accurate in terms of relative rounding error that influences generated samples.