P2P Storage Systems Research Articles

Decentralized and locality aware replication method for DHT-based P2P storage systems

Skip Graph, a type of DHT, plays an important role in P2P cloud storage applications, where nodes publicly or privately store, share, and access data. Nowadays P2P storage systems are widely using replication to support data availability, reliability, and maintainability. With replication, the main consideration is determining peers to replicate the data. Traditional replication algorithms are partially randomized and employ rigid assumptions about nodes’ distribution. This results in high access delay between nodes and their closest replicas, which degrades the system performance. We propose GLARAS , a dynamic and fully decentralized locality aware replication method for Skip Graph. In contrast to the traditional algorithms, which replicate based on strict assumptions about the distribution of nodes, GLARAS aims to approximate the underlying distribution by interacting with a very small subset of nodes and minimize the average access delay of replication accordingly. To ensure GLARAS performs at its best, we also propose a dynamic fully decentralized landmark-based locality aware name ID assignment namely LANS . This ensures that the nodes’ distances in the overlay and the underlying network are consistent with each other. Our extensive experiments and analysis results demonstrate that compared to the best existing decentralized locality aware replication, GLARAS improves the average access delay of public and private replications by about 13% and 17%, respectively. Likewise, in comparison to the best existing decentralized locality aware name ID assignment, LANS improves the locality awareness of name IDs and the end-to-end latency of search queries in Skip Graph with the gains of about 19% and 8%, respectively. The average replication’s access delay of a Skip Graph-based P2P storage system that employs GLARAS and LANS has an improvement gain of about 2.7 over the best state-of-the-art algorithms. Since Skip Graph is a DHT, any other DHT-based P2P storage service would benefit from our solution.

On the interplay between data redundancy and retrieval times in P2P storage systems

Peer-to-peer (P2P) storage systems aggregate spare storage resources from end users to build a large collaborative online storage solution. In these systems, however, the high levels of user churn—peers failing or leaving temporarily or permanently—affect the quality of the storage service and might put data reliability on risk. Indeed, one of the main challenge of P2P storage systems has traditionally been how to guarantee that stored data can always be retrieved within some time frame. To meet this challenge, existing systems store objects with high amounts of data redundancy, rendering data availability values close to 100%, which in turn ensure optimal retrieval times (only constrained by network limits). Unfortunately, this redundancy reduces the overall net capacity of the system and increases data maintenance costs. To alleviate these problems data redundancy can be reduced at the expense of lengthening retrieval times. The problem is that both the rewards and disadvantages of doing so are not well understood. In this paper we present a novel analytical framework that allows us to model retrieval times in P2P storage systems and describe the interplay between data redundancy and retrieval times for different churn patterns. Using availability traces from real P2P applications, we show that our framework provides accurate estimation of retrieval times in realistic environments.

P2P storage systems: Study of different placement policies

In a P2P storage system using erasure codes, a data block is encoded in many redundancy fragments. These fragments are then sent to distinct peers of the network. In this work, we study the impact of different placement policies of these fragments on the performance of storage systems. Several practical factors (easier control, software reuse, latency) tend to favor data placement strategies that preserve some degree of locality. We compare three policies: two of them are local, in which the data are stored in logical neighbors, and the other one, global, in which the data are spread randomly in the whole system. We focus on the study of the probability to lose a data block and the bandwidth consumption to maintain such redundancy. We use simulations to show that, without resource constraints, the average values are the same no matter which placement policy is used. However, the variations in the use of bandwidth are much more bursty under the local policies. When the bandwidth is limited, these bursty variations induce longer maintenance time and henceforth a higher risk of data loss. We then show that a suitable degree of locality could be introduced in order to combine the efficiency of the global policy with the practical advantages of a local placement. Additionally, we propose a new external reconstruction strategy that greatly improves the performance of local placement strategies. Finally, we give analytical methods to estimate the mean time to the occurrence of data loss for the three policies.

A low-cost, high-availability P2P storage scheme with regenerating code that combines caching methods

Peer-to-peer (P2P) storage systems are expected to be fast, fault-tolerant, scalable, and reliable. Thus, high data availability is an essential feature of P2P storage systems. In this article, we present a scheme that utilizes a regenerating code (RC) with caching to improve the performance of content services provided by P2P storage systems. Erasure coding is commonly employed to support high availability, but this technique requires the original file to produce redundant data. In contrast, RC solves the problem by collecting the encoded information. However, RC requires more peers to decode the blocks, which is difficult in a P2P environment because more peers must simultaneously remain active to hold the file blocks. Based on the RC, we store the information for a peer that has recently accessed blocks and utilized the data in the peer's LRU cache to increase the access performance and reduce the encoding cost. We carried out a series of experiments with different cache sizes under various levels of P2P availability. The results show that our scheme can outperform the traditional RC system in terms of access performance, allowing access at least 83% of the time, while also achieving a lower cost.

User-experience-based availability analysis model and its application in P2P storage systems

Data availability is one of the most important properties of peer-to-peer (P2P) storage systems. Availability analysis model and data placement are two key design choices. Users in P2P storage system are both providers and customers. This characteristic determines that the availability analysis must be user-centric, and thereby enhance the quality of service and decrease the system cost. The popular approach in recent studies is simple random placement with steady-state model, which has the following drawbacks: 1) It ignores the up/down patterns of nodes, whose availability is over-estimated or under-estimated at different periods of time. 2) It ignores the access patterns of users, so the availability perceived by users is hard to evaluate precisely. 3) It ignores the huge difference of nodes’ availability, thus leading to the absence of incentive. This paper proposes a novel user-experience-based availability model, which evaluates the availability of P2P storage system in terms of user experience, which can degenerate to traditional availability analysis model. Based on the new model, this paper proposes decentralized data placement algorithms for two typical P2P storage applications: “data sharing” and “personal backup”. By the trace-driven simulation, we prove that our methods can enhance the availability perceived by users greatly, reduce the variance of the availability dramatically and eliminate the nodes with low availability in data-sharing applications; meanwhile, it can provide different-level service to encourage users according to their contributions.

AutoProc: An automatic proactive replication scheme for P2P storage

Due to the prevalence of peer departures, P2P storage systems need to constantly generate replicas for high durability. However, few existing systems are able to select the right replication rate that balances replication cost and durability, e.g., minimizing replication cost without threatening the durability target. In this paper, we propose a method called AutoProc to automate the replication process in P2P storage systems. AutoProc employs two key techniques to enable cost-effective replication: correctly adapting replication rate with a reliability model, and replicating objects at the right order with a reliability indicator. Extensive simulations based on the real system failure trace show that AutoProc achieves the near-optimal performance.

Protector: A Probabilistic Failure Detector for Cost-Effective Peer-to-Peer Storage

Maintaining a given level of data redundancy is a fundamental requirement of peer-to-peer (P2P) storage systems—to ensure desired data availability, additional replicas must be created when peers fail. Since the majority of failures in P2P networks are transient (i.e., peers return with data intact), an intelligent system can reduce significant replication costs by not replicating data following transient failures. Reliably distinguishing permanent and transient failures, however, is a challenging task, because peers are unresponsive to probes in both cases. In this paper, we propose Protector, an algorithm that enables efficient replication policies by estimating the number of “remaining replicas” for each object, including those temporarily unavailable due to transient failures. Protector dramatically improves detection accuracy by exploiting two opportunities. First, it leverages failure patterns to predict the likelihood that a peer (and the data it hosts) has permanently failed given its current downtime. Second, it detects replication level across groups of replicas (or fragments), thereby balancing false positives for some peers against false negatives for others. Extensive simulations based on both synthetic and real traces show that Protector closely approximates the performance of a perfect “oracle” failure detector, and significantly outperforms time-out-based detectors using a wide range of parameters. Finally, we design, implement and deploy an efficient P2P storage system called AmazingStore by combining Protector with structured P2P overlays. Our experience proves that Protector enables efficient long-term data maintenance in P2P storage systems.

Towards a more accurate availability evaluation in peer-to-peer storage systems

Data availability evaluation is the crucial problem to build large-scale, high available peer-to-peer storage systems by governing many unreliable hosts. However, many recent studies equate host availability with data availability and calculate data availability based on the traditional definition of host availability. By analysing the trace of live P2P system, we argue that there is a significant difference between them and it is far more inaccurate and misleading to use host availability to evaluate data availability because of peers' incentives and rationalities. Firstly, to achieve a precise data availability evaluation, we present an evaluation model which combines peers' incentives and host availability together. Furthermore, we illustrate that the traditional definition of host availability has serious drawbacks because it ignores the effect of time-of-day and peers' different online pattern. So we improve the accuracy of our model by considering both factors. Finally, we introduce how to implement this model in practice and propose some of its applications to P2P storage systems.

MUREX

PurposeThe purpose of this paper is to propose MUREX, a mutable replica control scheme, to keep one‐copy equivalence for synchronous replication in structured peer‐to‐peer (P2P) storage systems.Design/methodology/approachFor synchronous replication in P2P networks, it is proper to adopt crash‐recovery as the fault model; that is, nodes are fail‐stop and can recover and rejoin the system after synchronizing their states with other active nodes. In addition to the state synchronization problem, the paper identifies other two problems to solve for synchronous replication in P2P storage systems. They are the replica acquisition and the replica migration problems.FindingsOn the basis of multi‐column read/write quorums, MUREX conquers the problems by the replica pointer, the on‐demand replica regeneration, and the leased lock techniques.Originality/valueThe paper proves the correctness of MUREX, analyzes and also simulates it in terms of communication cost and operation success rate.

Study on Durable Peer-to-Peer Storage Techniques

Peer-to-Peer (P2P) has been one of the most important architectures for Internet applications for its inherent scalability, fault tolerance, and high performance. The research of P2P storage systems is one of the hottest issues, and P2P storage system is regarded as one of the most promising P2P applications. However, to provide durable data storage is not trivial work and sets great barrier to real deployed systems. This paper surveys the P2P storage systems and techniques for durable storage. This paper first introduces the basic components of a durable P2P storage system and the advantages by using P2P architecture, and then introduces a series of techniques for durable storage including data redundancy, data placement, failure detection and data maintenance. Following the research framework, some typical P2P storage systems and their techniques are introduced. By a detailed comparison, the pros and cons of the techniques are discussed. In the end, the problems in current research and some future research issues are outlined.

Kosha: A Peer-to-Peer Enhancement for the Network File System

The storage needs of modern scientific applications are growing exponentially, and designing economical storage solutions for such applications – especially in Grid environments – is an important research topic. This work presents Kosha, a system that aims to harvest redundant storage space on cluster nodes and user desktops to provide a reliable, shared file system that acts as a large distributed storage. Kosha utilizes peer-to-peer (p2p) mechanisms to enhance the widely-used Network File System (NFS). P2P storage systems provide location transparency, mobility transparency, load balancing, and file replication – features that are not available in NFS. On the other hand, NFS provides hierarchical file organization, directory listings, and file permissions, which are missing from p2p storage systems. By blending the strengths of NFS and p2p storage systems, Kosha provides a low overhead storage solution. Our experiments show that compared to unmodified NFS, Kosha introduces a 3.3% fixed overhead and 4.5% additional overhead as nodes are increased from two to sixteen. For larger number of nodes, the additional overhead increases slowly. Kosha achieves load balancing in distributed directories, and guarantees $99.99\%$ or better file availability.