RAID-6 Systems Research Articles

Hard drive failure prediction using Decision Trees

This paper proposes two hard drive failure prediction models based on Decision Trees (DTs) and Gradient Boosted Regression Trees (GBRTs) which perform well in prediction performance as well as stability and interpretability. The models are evaluated on a real-world dataset containing 121,698 drives in total. Experimental results show the DT model predicts over 93% of failures at a false alarm rate under 0.01%, and the GBRT model can achieve about 90% failure detection rate without any false alarms. Moreover, the GBRT model evaluates drive health (or fault probability) which provides a quantitative indicator of failure urgency. This enables operators to allocate system resources accordingly for pre-warning migrations while maintaining the quality of user services.Aiming at practical application of prediction models, we test the models on another real-world dataset with different drive models, on a real-world hybrid dataset with multiple drive models, and on several datasets containing fewer drives. Both prediction models show steady prediction performance, with high failure detection rates (80% to 96%) and low false alarm rates (0.006% to 0.31%). We also implement a reliability model for RAID-6 systems with proactive fault tolerance and show that the proposed models can significantly improve the reliability and/or reduce construction and maintenance cost of large-scale storage systems.

RAIDShield

Modern storage systems orchestrate a group of disks to achieve their performance and reliability goals. Even though such systems are designed to withstand the failure of individual disks, failure of multiple disks poses a unique set of challenges. We empirically investigate disk failure data from a large number of production systems, specifically focusing on the impact of disk failures on RAID storage systems. Our data covers about one million SATA disks from six disk models for periods up to 5 years. We show how observed disk failures weaken the protection provided by RAID. The count of reallocated sectors correlates strongly with impending failures. With these findings we designed RAIDS hield , which consists of two components. First, we have built and evaluated an active defense mechanism that monitors the health of each disk and replaces those that are predicted to fail imminently. This proactive protection has been incorporated into our product and is observed to eliminate 88% of triple disk errors, which are 80% of all RAID failures. Second, we have designed and simulated a method of using the joint failure probability to quantify and predict how likely a RAID group is to face multiple simultaneous disk failures, which can identify disks that collectively represent a risk of failure even when no individual disk is flagged in isolation. We find in simulation that RAID-level analysis can effectively identify most vulnerable RAID-6 systems, improving the coverage to 98% of triple errors. We conclude with discussions of operational considerations in deploying RAIDS hield more broadly and new directions in the analysis of disk errors. One interesting approach is to combine multiple metrics, allowing the values of different indicators to be used for predictions. Using newer field data that reports an additional metric, medium errors , we find that the relative efficacy of reallocated sectors and medium errors varies across disk models, offering an additional way to predict failures.

A HIERARCHICAL MARKOV RELIABILITY MODEL FOR DATA STORAGE SYSTEMS WITH MEDIA SELF-RECOVERY

A Redundant Array of Independent Disks (RAID) system with n disks or hard drives can be modeled as a k-out-of-n repairable system for which at least k operational disks are required. This paper proposes a hierarchical Markov model to estimate the reliability of RAID systems. The method encompasses Markov models for evaluating the reliability of individual disks at the lower level and the redundant model for evaluating the reliability of entire RAID at the system level. Both hardware and media failures are considered, and the media failures can be potentially recovered via the disk self-restoration mechanism. The system mean-time-to-data-loss is also derived and results are compared to those estimated from system-based Markov models. The major contribution of this work is that the reliability for RAID systems is approached by combining the redundant modeling technique with the single disk-based Markov model, thus simplifying the computational efforts. The proposed method is applied on RAID-5 and RAID-6 systems to demonstrate the applicability and performance of the new model.

Triple-Parity RAID and Beyond

How much longer will current RAID techniques persevere? The RAID levels were codified in the late 1980s; double-parity RAID, known as RAID-6, is the current standard for high-availability, space-efficient storage. The incredible growth of hard-drive capacities, however, could impose serious limitations on the reliability even of RAID-6 systems. Recent trends in hard drives show that triple-parity RAID must soon become pervasive. In 2005, Scientific American reported on Kryder’s law, which predicts that hard-drive density will double annually. While the rate of doubling has not quite maintained that pace, it has been close.

The Raid-6 Liber8Tion Code

Large centralized and networked storage systems have grown to the point where the single fault tolerance provided by RAID-5 is no longer enough. RAID-6 storage systems protect k disks of data with two parity disks so that the system of k + 2 disks may tolerate the failure of any two disks. Coding techniques for RAID-6 systems are varied, but an important class of techniques are those with minimum density, featuring an optimal combination of encoding, decoding and modification complexity. The word size of a code has an impact on both how the code is laid out on each disk's sectors and how large k can be. Word sizes which are powers of two are especially important, since they fit precisely into file system blocks. Minimum density codes exist for many word sizes with the notable exception of eight. This paper fills that gap by describing a new code called The RAID-6 Liber8tion Code for this important word size. The description includes performance properties as well as details of the discovery process.

A new intra-disk redundancy scheme for high-reliability RAID storage systems in the presence of unrecoverable errors

Today's data storage systems are increasingly adopting low-cost disk drives that have higher capacity but lower reliability, leading to more frequent rebuilds and to a higher risk of unrecoverable media errors. We propose an efficient intradisk redundancy scheme to enhance the reliability of RAID systems. This scheme introduces an additional level of redundancy inside each disk, on top of the RAID redundancy across multiple disks. The RAID parity provides protection against disk failures, whereas the proposed scheme aims to protect against media-related unrecoverable errors. In particular, we consider an intradisk redundancy architecture that is based on an interleaved parity-check coding scheme, which incurs only negligible I/O performance degradation. A comparison between this coding scheme and schemes based on traditional Reed--Solomon codes and single-parity-check codes is conducted by analytical means. A new model is developed to capture the effect of correlated unrecoverable sector errors. The probability of an unrecoverable failure associated with these schemes is derived for the new correlated model, as well as for the simpler independent error model. We also derive closed-form expressions for the mean time to data loss of RAID-5 and RAID-6 systems in the presence of unrecoverable errors and disk failures. We then combine these results to characterize the reliability of RAID systems that incorporate the intradisk redundancy scheme. Our results show that in the practical case of correlated errors, the interleaved parity-check scheme provides the same reliability as the optimum, albeit more complex, Reed--Solomon coding scheme. Finally, the I/O and throughput performances are evaluated by means of analysis and event-driven simulation.