An Accurate and Invertible Sketch for Super Spread Detection

Abstract

Super spread detection has been widely applied in network management, recommender systems, and cyberspace security. It is more complicated than heavy hitter owing to the requirement of duplicate removal. Accurately detecting a super spread in real-time with small memory demands remains a nontrivial yet challenging issue. The previous work either had low accuracy or incurred heavy memory overhead and could not provide a precise cardinality estimation. This paper designed an invertible sketch for super spread detection with small memory demands and high accuracy. It introduces a power-weakening increment strategy that creates an environment encouraging sufficient competition at the early stages of discriminating a super spread and amplifying the comparative dominance to maintain accuracy. Extensive experiments have been performed based on actual Internet traffic traces and recommender system datasets. The trace-driven evaluation demonstrates that our sketch actualizes higher accuracy in super spread detection than state-of-the-art sketches. The super spread cardinality estimation error is 2.6–19.6 times lower than that of the previous algorithms.