Cuckoo Counter: A Novel Framework for Accurate Per-Flow Frequency Estimation in Network Measurement

Abstract

Per-flow frequency estimation plays a fundamental role in network measurement. As a probabilistic data structure, sketch has been extensively investigated and used for per-flow frequency estimation, but most sketch-based proposals in previous literatures cannot achieve high accuracy and high speed simultaneously. Moreover, because each insertion to a sketch causes increment in multiple entries, the over-estimation error will accumulate quickly over time. In this paper, we propose Cuckoo Counter, a compact and accurate framework for per-flow frequency estimation, which employs three novel ideas: (1)kicking out conflicting flows instead of using multiple entries counts to improve accuracy; (2)using different sizes of entries to insulate mice flows from elephant flows, which can handle the skewed data streams efficiently and improve memory utilization; (3) a Cuckoo-like replacement strategy for mice flows, so as to maintain accurate records for elephant flows. To verify the effectiveness and efficiency of our framework, we compared it with two well-known sketches as well as the recent proposed Augmented sketch and Pyramid sketch. Extensive experimental results on three different types of test datasets show that Cuckoo Counter outperforms these sketches considerably.