On Load-Balancing Algorithm for Distributed Data Stream Management Systems

Abstract

In distributed data stream management system, data streams are processed by many computers cooperatively. When input data streams fluctuate tempestuously, overload of some nodes may influence the ability of data processing about system. To solve this practical problem, the properties of the system were analysed. A multi-layer overlay based load balancing algorithm was proposed in this paper, which can balance load of tightly-coupled distributed data stream management system in Internet. This algorithm collects node-load and send target-load using virtual tree-structure, it also maps address by virtual circle-structure to match the load distribution information early so the time of load-move was decreased. This multi-layer structure can response quickly the effect caused by the departure or crumple of node, which makes the system refresh easily. At last we demonstrate properties of the algorithm using simulation and experiments.