Graph-based model and algorithm for minimising big data movement in a cloud environment

Abstract

In this paper, we discuss load balancing and data placement strategies in cloud environments. The main goal in data placement strategies is to improve the overall performance through the reduction of data movements among the participating datacentres. Load balancing and efficient data placement on cloud systems are critical problems that are difficult to simultaneously cope with. In this context, we propose a threshold-based load balancing algorithm, which first balances the load between datacentres, and afterwards minimises the overhead of data exchanges. It is divided into three phases. First, the dependencies between the datasets are identified. Second, the load threshold of each datacentre is estimated based on the processing speed and the storage capacity. Third, the load balancing between the datacentres is managed through the threshold parameters. Our experimental results show that our approach can efficiently reduce the frequency of data movement and keep a good load balancing between the datacentres.