Tabular Interpolation Approach Based on Stable Random Projection for Estimating Empirical Entropy of High-Speed Network Traffic

Abstract

The empirical entropy of the network flow attributes is an essential measure for identifying anomalous network traffic. However, computing the exact entropy values for high-speed networks in real-time is computationally expensive. Accordingly, the present study replaces the complex computations of existing stable random projection methods for entropy estimation with a simple table lookup procedure. Notably, the size of the lookup table is reduced through a piece-wise linear interpolation heuristic in order to facilitate the implementation of the proposed scheme in resource-constrained pipeline environments. The proposed architecture enables entropy estimation to be performed using both the Log-Mean Estimator (LME) method and the New Estimator of Compressed Counting (NECC) algorithm reported in the literature. The feasibility of the proposed approach is verified empirically using both real-world network traffic traces and synthetic data streams. Moreover, the practical applicability is demonstrated via stream-based implementation in the programmable data planes of the NetFPGA-Plus framework and a Tofino P4 switch, respectively. The results indicate that the proposed tabulation-based entropy estimation scheme allows minimum-sized Ethernet frames to be processed with a wire speed of up to several hundred gigabits per second.