Optimally detecting uniformly-distributed $\ell_2$ heavy hitters in data streams

Abstract

Given a stream $x_1,x_2,\dots,x_n$ of items from a Universe $U$ of size poly$(n)$, and a parameter $\epsilon>0$, an item $i\in U$ is said to be an $\ell_2$ heavy hitter if its frequency $f_i$ in the stream is at least $\sqrt{\epsilon F_2}$, where $F_2={\sum_{i\in U} f_i^2}$. Efficiently detecting such heavy hitters is a fundamental problem in data streams and has several applications in both theory and in practice. The classical $\mathsf{CountSketch}$ algorithm due to Charikar, Chen, and Farach-Colton [2004], was the first algorithm to detect $\ell_2$ heavy hitters using $O\left(\frac{\log^2 n}{\epsilon}\right)$ bits of space, and their algorithm is optimal for streams with deletions. A work due to Braverman, Chestnut, Ivkin, Nelson, Wang, and Woodruff [2017] gave the $\mathsf{BPTree}$ algorithm which detects $\ell_2$ heavy hitters in insertion-only streams using only $O\left(\frac{\log(1/\epsilon)}{\epsilon}\log n \right)$ space. Note that any algorithm requires at least $\Omega\left(\frac{1}{\epsilon} \log n\right)$ space to output $O(1/\epsilon)$ heavy hitters in the worst case. While $\mathsf{BPTree}$ achieves optimal space bound for constant $\epsilon$, their bound could be sub-optimal for $\epsilon=o(1)$. For $\textit{random order}$ streams, where the stream elements can be adversarial but their order of arrival is uniformly random, Braverman, Garg, and Woodruff [2020] showed that it is possible to achieve the optimal space bound of $O\left(\frac{1}{\epsilon} \log n\right)$ for every $\epsilon = \Omega\left(\frac{1}{2^{\sqrt{\log n}}}\right)$. In this work, we generalize their result to $\textit{partially random order}$ streams where only the heavy hitters are required to be uniformly distributed in the stream. We show that it is possible to achieve the same space bound, but with an additional assumption that the algorithm is given a constant approximation to $F_2$ in advance.