Abstract
Owing to the high availability and space-efficiency of erasure codes, they have become the de facto standard to provide data durability in large scale distributed storage systems. The update-intensive workloads of erasure codes lead to a large amount of data transmission and I/O. As a result, it becomes a major challenge to reduce the amount of data transmission and optimize the use of existing network resources so that the update efficiency of the erasure codes could be improved. However, very little research has been done to optimize the update efficiency of the erasure codes under multiple QoS metrics. In this paper, our proposed update scheme, the Ant Colony Optimization based multiple data nodes Update Scheme (ACOUS) employs a two-stage rendezvous data update procedure to optimize the multiple data nodes updates. Specifically, the two-stage rendezvous data update procedure performs the data delta collection and the parity delta distribution based on a multi-objective update tree which is built by the ant colony optimization routing algorithm. Under typical data center network topologies, extensive experimental results show that, compared to the traditional TA-Update scheme, our scheme is able to achieve a 26% to 37% reduction of update delay with convergence guarantee at the cost of negligible computation overhead.
Highlights
INTRODUCTIONY. Hu et al.: Ant Colony Optimization Based Data Update Scheme for Distributed Erasure-Coded Storage Systems codes, such as Azure [17] and CodFS [14], adopt a log-based data update or a hybrid of in-place data updates and logbased parity updates to reduce I/Os by sequentially appending updates
Our proposed Ant Colony Optimization based multiple data nodes Update Scheme (ACOUS) adopts a two-stage rendezvous data update procedure to optimize the multiple data nodes updates and routing, which performs the data delta collection and the parity delta distribution based on an update tree that is built by the Multi-objective Ant Colony Optimization Update routing algorithm (MACOU)
3) We develop a prototype and perform extensive experiments to evaluate the performance of the ACOUS under typical data center network topologies
Summary
Y. Hu et al.: Ant Colony Optimization Based Data Update Scheme for Distributed Erasure-Coded Storage Systems codes, such as Azure [17] and CodFS [14], adopt a log-based data update or a hybrid of in-place data updates and logbased parity updates to reduce I/Os by sequentially appending updates. Our proposed Ant Colony Optimization based multiple data nodes Update Scheme (ACOUS) adopts a two-stage rendezvous data update procedure to optimize the multiple data nodes updates and routing, which performs the data delta collection and the parity delta distribution based on an update tree that is built by the Multi-objective Ant Colony Optimization Update routing algorithm (MACOU). Our main contributions are summarized as follows: 1) To the authors’ best knowledge, the ACOUS is the first work to provide a thorough study of the multiple data nodes updates and routing under heterogeneous erasure-coded storage systems, in view of the unequal memory throughput and link bandwidth across nodes.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have