Towards a delivery scheme for speedup of data backup in distributed storage systems using erasure codes

Abstract

Distributed storage systems, built on peer-to-peer networks, can provide large-scale data storage and high data reliability by redundancy. Data backup is the process to store data into a set of redundant storage nodes. Rapid completion of such a process is very critical to maintain system performance. In traditional data backup in distributed systems based on erasure codes, star-structured scheme is used, in which each redundant block is just sent to each target storage node from the source node directly, so the storage throughput and delay are limited by the bottleneck bandwidth, due to bandwidth heterogeneity. The recent “in-network” redundancy generation scheme uses locally repairable property of self-repairing codes to speed up data backup. However, such kind of code does not own maximum distance separable property, thus does not achieve optimal storage efficiency. We still lack a fast backup scheme in distributed systems based on general erasure coding. To this end, we proposed that instead of only focusing on bandwidths between the source node and target nodes, the bandwidths between target storage nodes should be fully taken into account. In our scheme, each redundant data block is divided into some parts according to different proportions and each part of the block is sent to the target storage node via other different storage nodes. The benefit is that spare bandwidths between target storage nodes are used to reduce backup time. We further show how this process can be modeled and derive a formula about the final backup time. We can achieve minimum backup time by solution for classical quadratic programming problem. We conduct both numerical analysis and experimental study. Our experiments shows, the delay reduces 59 %, compared with common star-structured scheme. Meanwhile, the throughput is increased significantly in backup process.