Study on Big Data Center Traffic Management Based on the Separation of Large-Scale Data Stream

Abstract

The network of traditional data center has been usually designed and constructed for the provision of user's equal access of data centre's resource or data. Therefore, network administrators have a strong tendency to manage user traffic from the viewpoint that the traffic has a similar size and characteristics. But, the emersion of big data begins to make data centers have to deal with 1015 byte-data transfer at once. Such a big data transfer can cause problems in network traffic management in the existed data center. And, the tiered network architecture of the legacy data center magnifies the magnitude of the problems. One of the well-known big data in science is from large hadron collider such as LHC in Swiss CERN. CERN LHC generates multi-peta byte data per year. From our experience of CERN data service, this paper showed the impact of network traffic affected by large-scale data stream using NS2 simulation, and then, suggested the evolution direction based on separating of large-scale data stream for the big data center's network architecture.