SuperSketch: A Multi-Dimensional Reversible Data Structure for Super Host Identification

Xuyang Jing,Witold Pedrycz,Hui Han,Zheng Yan

doi:10.1109/tdsc.2021.3072295

Abstract

Facing big network traffic data, effective data compression becomes crucially important and urgently needed for estimating host cardinalities and identifying super hosts. However, the current literature confronts several challenges: incapability of simultaneously measuring various types of host cardinalities and inability to efficiently reconstruct super host addresses. To address these challenges, in this article, we propose a novel sketch data structure, named SuperSketch, to simultaneously measure multiple types of host cardinalities with the purpose of efficiently identifying super hosts. SuperSketch has two significant characteristics: multi-dimensionality and reversibility. The multi-dimensionality makes SuperSketch capable of simultaneously measuring Source Cardinality, Destination Cardinality, and Destination Port Cardinality. The reversibility allows SuperSketch to accurately and quickly reconstruct the original addresses of super hosts once they are identified. We conduct both theoretical analysis and performance evaluation based on real-world network traffic. Experimental results show that SuperSketch achieves outstanding performance for multi-cardinality measurement, super host identification, and host address reconstruction.

Highlights

A CCURATE and timely network traffic measurement is an essential part of network security management and network information forensics, e.g., host cardinality measurement [1]-[5], flow size measurement [6]-[10], abnormal behavior measurement [11]-[15], and persistent spread measurement [16], [17]
With the purpose of identifying network anomalies, we only focus on the incremental super changers whose Destination Cardinality (DC)/Destination Port Cardinality (DPC) measured in current time interval are much larger than the cardinality calculated in previous time interval because some scanning activities will be conducted before launching network attacks
SuperSketch has three parameters, namely the number of hash functions N that highly determines the processing time, P that is the approximate length of L1 and L2, and U that is the approximate length of L3

Summary

Introduction

A CCURATE and timely network traffic measurement is an essential part of network security management and network information forensics, e.g., host cardinality measurement [1]-[5], flow size measurement [6]-[10], abnormal behavior measurement [11]-[15], and persistent spread measurement [16], [17]. Among these applications, host cardinality measurement is a distinctive task. The last is Source Cardinality (SC) that is the number of distinct source addresses that a host is connected with

Methods

Results

Discussion

Conclusion