Parallelizing weighted frequency counting in high-speed network monitoring

Abstract

Identifying frequent items in high-speed network is important for a variety of network applications ranging from traffic engineering to anomaly detection such as detection of denial of service attacks. To deal with high packet arrival rate, it is desirable that such systems are able to support very high update throughput. The advent of multi-core processors calls for efficient parallel designs which can effectively utilize the parallelism of the multi-cores. In this paper, we address the problem of parallelizing weighted frequency counting in the context of multi-core processors. We discuss the challenges in designing an efficient parallel system. Our evaluation and analysis reveals that the naive fine-grained lock design results in excessive overhead and wait, which in turn leads to severe performance degradation in multi-core architectures. Based on our analysis, we propose a novel method: precision integrated method (PRIM). PRIM makes use of the temporal imprecision concept to significantly reduce the merge overhead at the cost of relatively large memory space used. Both the theoretical analysis and real traffic experiments demonstrate that PRIM delivers almost linear speedup.