Memory efficient distributed sliding super point cardinality estimation by GPU

Abstract

Super point is a kind of special host in the network which contacts with huge of other hosts. Estimating its cardinality, the number of other hosts contacting with it, plays important roles in network management. But all of existing works focus on discrete time window super point cardinality estimation which has great latency and ignores many measuring periods. Sliding time window measures super point cardinality in a finer granularity than that of discrete time window but also more complex. This paper firstly introduces an algorithm to estimate super point cardinality under sliding time window from distributed edge routers. This algorithm's ability of sliding super point cardinality estimating comes from a novel method proposed in this paper which can record the time that a host appears. Based on this method, two sliding cardinality estimators, sliding rough estimator and sliding linear estimator, are devised for super points detection and their cardinalities estimation separately. When using these two estimators together, the algorithm consumes the smallest memory with the highest accuracy. This sliding super point cardinality algorithm can be deployed in distributed environment and acquire the global super points' cardinality by merging estimators of distributed nodes. Both of these estimators could process packets parallel which makes it becom possible to deal with high speed network in real time by GPU. Experiments on a real world traffic show that this algorithm have the highest accuracy and the smallest memory comparing with others when running under discrete time window. Under sliding time window, this algorithm also has the same performance as under discrete time window.