Streaming with Catalytic Memory

Abstract

We introduce a streaming model that uses both catalytic and regular memory. In this model, we show how to exactly compute the frequency moments using a logarithmic number of bits of regular memory and a polynomial number of bits of catalytic memory. More generally, we show how to compute arbitrary polynomials of the item frequencies exactly within the same space bounds. As an application, we obtain catalytic streaming algorithms that exactly compute the number of distinct elements in a stream, count the number of triangles (or any other small subgraph) in a graph whose edges arrive in a stream, and identify heavy hitters. Our algorithms for frequency moments perform a constant number of passes over the stream, and for polynomial evaluation, we require one more pass than the degree of the polynomial. By relating our catalytic streaming model to the catalytic communication model introduced in Pyne et al., we show that catalytic memory is not useful for any one-pass streaming algorithm. For lower bounds on multipass streaming algorithms, the impossibility results of Pyne et al. are not strong enough. However, using a different technique, we show that under certain natural restrictions, no catalytic streaming algorithm can compute the second frequency moment in fewer than three passes. This definition of the restricted class of two-pass algorithms then guides us in the design of a two-pass algorithm for computing the second moment exactly that circumvents these restrictions and breaks the three-pass barrier.