ESDU: An elastic stripe-based delta update method for erasure-coded cross-data center storage systems

Abstract

The erasure-coded cross-data center storage system can achieve high disaster tolerance and low redundancy. But as it has large cross-data center update traffic, its data update time is long. In erasure-coded storage systems, each data object is sequentially divided into several stripes, and each stripe consists of several data packets. When erasure-coded stripes do not undergo insertion or deletion, existing work can effectively reduce cross-data center update traffic by performing delta updates—delta update methods can update the old stripe without transferring matched packets that are new and old stripes' duplicate packets with the same offset-within-stripe. However, because existing delta update methods' stripe size is fixed, when a stripe undergoes insertion or deletion, its subsequent stripes' duplicate packets' offset-within-stripe will change. In this scenery, the matched packet number is small, resulting in large cross-data center update traffic. This paper proposes an elastic stripe-based delta update method for erasure-coded cross-data center storage systems (ESDU). Under insertion or deletion, ESDS tries to avoid duplicate packets' offset-within-stripe changing (i.e., maximizing matched packets) by adjusting the stripes' size flexibly according to the duplicate packet locating result. So, it can reduce cross-data center traffic. Moreover, ESDU can optimize stripes' update topology based on the location information of storage nodes to reduce cross-data center update traffic further. In addition, we implement an erasure-coded cross-data center storage system adopting ESDU, called ECESD. Experiments with the workloads derived from EduCoder's real-world trace show that compared with the existing erasure-coded cross-data center storage system adopting the fixed stripe-based delta update method, ECESD reduces average update time by 89.6%. Moreover, compared with a replication-based storage system with a delta update method (HadoopRsync), ECESD achieves an 8.3% shorter update time and much smaller redundancy.