Virtual indexing based methods for estimating node connection degrees

Abstract

It is difficult to accurately measure node connection degrees for a high speed network, since there is a massive amount of traffic to be processed. In this paper, we present a new virtual indexing method for estimating node connection degrees for high speed links. It is based on the virtual connection degree sketch (VCDS) where a compact sketch of network traffic is built by generating multiple virtual bitmaps for each network node. Each virtual bitmap consists of a fixed number of bits selected randomly from a shared bit array by a new method for recording the traffic flows of the corresponding node. The shared bit array is efficiently utilized by all nodes since every bit is shared by the virtual bitmaps of multiple nodes. To reduce the “noise” contaminated in a node’s virtual bitmaps due to sharing, we propose a new method to generate the “filtered” bitmap used to estimate node connection degree. Furthermore, we apply VCDS to detect super nodes often associated with traffic anomalies. Since VCDS need a large amount of extra memory to store node addresses, we also propose a new data structure, the reversible virtual connection degree sketch, which identifies super node addresses analytically without the need of extra memory space but at a small increase in estimation error. Furthermore we combine the VCDS and RVCDS based methods with a uniform flow sampling technique to reduce memory complexities. Experiments are performed based on the actual network traffic and testing results show that the new methods are more memory efficient and more accurate than existing methods.