Design of Network Analysis System Based on Stream Computing

Abstract

The functions of the network analysis system include detection and analysis of network data stream. According to the results of the network analysis, we monitor the network accident and avoid the security risks. This can improve the network performance and increase the network availability. As the data flow in the network is constantly produced, the biggest characteristic of network analysis system is that it is a real-time system. Because of the high requirements of the network data analysis and network fault processing, the system requires very high processing efficiency of the real time data of network. Stream computing is a technique specifically for processing real-time data streams. Its idea is that the value of the data is reduced with the lapse of time, so as long as the data appearing, it must be processed as soon as possible. So we use the technology of stream computing to design network analysis system to meet the needs of real-time capability. Moreover, the stream computing framework has been widely welcomed in the field because of its good expansibility, ease of use and flexibility. In this paper, firstly, we introduce the characteristics of the data processing based on stream computing and the traditional data processing separately. We point out their difference and introduce the technique of stream computing. Then, we introduce the architecture of network analysis system designed base on the technique of stream computing. The architecture includes two main components that are logic processing layer and communication layer. We describe the characteristics of each component and functional characteristics in detail, and we introduce the system load balancing algorithm. Finally, by experiments, we verify the effectiveness of the system’s characteristics of dynamic expansion and load balancing.