Randomized Admission Policy for Efficient Top-k, Frequency, and Volume Estimation

Abstract

Network management protocols often require timely and meaningful insight about per flow network traffic. This paper introduces Randomized Admission Policy (RAP) –a novel algorithm for the frequency , top-k , and byte volume estimation problems, which are fundamental in network monitoring. We demonstrate space reductions compared to the alternatives, for the frequency estimation problem, by a factor of up to 32 on real packet traces and up to 128 on heavy-tailed workloads. For top- $k$ identification, RAP exhibits memory savings by a factor of between 4 and 64 depending on the workloads’ skewness. These empirical results are backed by formal analysis, indicating the asymptotic space improvement of our probabilistic admission approach. In Addition, we present d-way RAP , a hardware friendly variant of RAP that empirically maintains its space and accuracy benefits.