Disk Failure Prediction Research Articles

SiaDFP: A Disk Failure Prediction Framework Based on Siamese Neural Network in Large-Scale Data Center

SiaDFP: A Disk Failure Prediction Framework Based on Siamese Neural Network in Large-Scale Data Center

Disk Failure Prediction based on Multi-layer Domain Adaptive Learning

Large scale data storage is susceptible to failure. As disks are damaged and replaced, traditional machine learning models, which rely on historical data to make predictions, struggle to accurately predict disk failures. This paper presents a novel method for predicting disk failures by leveraging multi-layer domain adaptive learning techniques. First, disk data with numerous faults is selected as the source domain, and disk data with fewer faults is selected as the target domain. A training of the feature extraction network is performed with the selected origin and destination domains. The contrast between the two domains facilitates the transfer of diagnostic knowledge from the domain of source and target. According to the experimental findings, it has been demonstrated that the proposed technique can generate a reliable prediction model and improve the ability to predict failures on disk data with few failure samples.

Disk failure prediction based on association analysis and SSA-LSTM

Hard disk is the main storage device for cloud service, and there always contain massive disks deployed in a data center. Disk failure occur frequently in data center, which may lead to data loss and other disasters, so there have urgent needs for a failure prediction method of hard disk so as to ensure service reliability. This paper proposes a temporal prediction model based on LSTM. Firstly, the SMART data of the disk is analyzed, and the Pearson correlation coefficient is used to analyze the correlation between the monitoring time series data of the faulty disk and the normal disk, and the monitoring index with the lowest correlation is selected as the fault feature; secondly, for the problem of serious imbalance of positive and negative samples in the SMART dataset, the SMOTEENN algorithm is introduced for oversampling to obtain a balanced dataset of positive and negative samples. The proposed method improves accuracy by 8.268% and F1-score by 8.657% compared to the conventional method.

Optimizing Efficiency of Machine Learning Based Hard Disk Failure Prediction by Two-Layer Classification-Based Feature Selection

Predicting hard disk failure effectively and efficiently can prevent the high costs of data loss for data storage systems. Disk failure prediction based on machine learning and artificial intelligence has gained notable attention, because of its good capabilities. Improving the accuracy and performance of disk failure prediction, however, is still a challenging problem. When disk failure is about to occur, the time is limited for the prediction process, including building models and predicting. Faster training would promote the efficiency of model updates, and late predictions not only have no value but also waste resources. To improve both the prediction quality and modeling timeliness, a two-layer classification-based feature selection scheme is proposed in this paper. An attribute filter calculating the importance of attributes was designed, to remove attributes insensitive to failure identification, where importance is gained based on the idea of classification tree models. Furthermore, by determining the correlation between features based on the correlation coefficient, an attribute classification method is proposed. In experiments, the models of machine learning and artificial intelligence were applied, and they included naïve Bayesian, random forest, support vector machine, gradient boosted decision tree, convolutional neural networks, and long short-term memory. The results showed that the proposed technique could improve the prediction accuracy of ML/AI-based hard disk failure prediction models. Specifically, utilizing random forest and long short-term memory with the proposed technique showed the best accuracy. Meanwhile, the proposed scheme could reduce training and prediction latency by 75% and 83%, respectively, in the best case compared with the baseline methods.

SPAE: Lifelong disk failure prediction via end-to-end GAN-based anomaly detection with ensemble update

Disk failure prediction aims to predict upcoming disk failures in advance for high data reliability. There are numerous supervised machine learning methods that are successful in predicting disk failure using SMART properties as input. However, these approaches heavily rely on a substantial number of annotated failed disks, resulting in degraded prediction performance caused by scarce failed disks at the beginning, also known as the cold start problem. Inspired by the success achieved in Generative Adversarial Network (GAN) based anomaly detection, this paper translates disk failure prediction into an anomaly detection problem. Specifically, we developed a Semi-supervised method for lifelong disk failure Prediction via Adversarial training and Ensemble update, called SPAE. The advantage of SPAE over existing supervised approaches is that SPAE can train the prediction model using only healthy disks, avoiding the cold start problem. Furthermore, SPAE can be updated using ensemble learning on emerging failed disks to resist the model aging problem. Compared to state-of-the-art methods using supervised machine learning on real-world datasets, SPAE predicts disk failures with higher accuracy for the full lifetime of models, i.e., both the startup period and the long-term usage.

Hard Disk Failure Prediction Based on Blending Ensemble Learning

As the most widely used storage device today, hard disks are efficient and convenient, but the damage incurred in the event of a failure can be very significant. Therefore, early warnings before hard disk failure, allowing the stored content to be backed up and transferred in advance, can reduce many losses. In recent years, an endless stream of research on the prediction of hard disk failure prediction has emerged. The detection accuracy of various methods, from basic machine learning models, such as decision trees and random forests, to deep learning methods, such as BP neural networks and recurrent neural networks, has also been improving. In this paper, based on the idea of blending ensemble learning, a novel failure prediction method combining machine learning algorithms and neural networks is proposed on the publicly available BackBlaze hard disk datasets. The failure prediction experiment is conducted only with S.M.A.R.T., that is, the learned characteristics collected by self-monitoring analysis and reporting technology, which are internally counted during the operation of the hard disk. The experimental results show that this ensemble learning model is able to outperform other independent models in terms of evaluation criterion based on the Matthews correlation coefficient. Additionally, through the experimental results on multiple types of hard disks, an ensemble learning model with high performance on most types of hard disks is found, which solves the problem of the low robustness and generalization of traditional machine learning methods and proves the effectiveness and high universality of this method.

StreamDFP: A General Stream Mining Framework for Adaptive Disk Failure Prediction

We explore machine learning for accurately predicting imminent disk failures and hence providing proactive fault tolerance for modern large-scale storage systems. Current disk failure prediction approaches are mostly offline and assume that the disk logs required for training learning models are available a priori. However, disk logs are often continuously generated as an evolving data stream, in which the statistical patterns vary over time (also known as concept drift). Such a challenge motivates the need of online techniques that perform training and prediction on the incoming stream of disk logs in real time, while being adaptive to concept drift. We first measure and demonstrate the existence of concept drift on various disk models in production. Motivated by our study, we design <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">StreamDFP</small> , a general stream mining framework for disk failure prediction with concept-drift adaptation based on three key techniques, namely online labeling, concept-drift-aware training, and general prediction, with a primary objective of supporting various machine learning algorithms. We extend <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">StreamDFP</small> to support online transfer learning for minority disk models with concept-drift adaptation. Our evaluation shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">StreamDFP</small> improves the prediction accuracy significantly compared to without concept-drift adaptation under various settings, and achieves reasonably high stream processing performance.

Retracted: Convolution‐LSTM‐Based Mechanical Hard Disk Failure Prediction by Sensoring S.M.A.R.T. Indicators

Retracted: Convolution‐LSTM‐Based Mechanical Hard Disk Failure Prediction by Sensoring S.M.A.R.T. Indicators

Fast Proactive Repair in Erasure-Coded Storage: Analysis, Design, and Implementation

Erasure coding offers a storage-efficient redundancy mechanism for maintaining data availability guarantees in large-scale storage clusters, yet it also incurs high performance overhead in failure repair. Recent developments in accurate disk failure prediction allow soon-to-fail (STF) nodes to be repaired in advance, thereby opening new opportunities for accelerating failure repair in erasure-coded storage. To this end, we present a fast proactive repair solution called FastPR, which carefully couples two repair methods, namely migration (i.e., relocating the chunks of an STF node) and reconstruction (i.e., decoding the chunks of an STF node through erasure coding), so as to fully parallelize the repair operation across the storage cluster. FastPR solves a bipartite maximum matching problem and schedules both migration and reconstruction in a parallel fashion. We show that FastPR significantly reduces the repair time over the baseline repair approaches for both Reed-Solomon codes and Azures Local Reconstruction Codes via mathematical analysis, large-scale simulation, and Amazon EC2 experiments.

A Disk Failure Prediction Method Based on Active Semi-supervised Learning

Disk failure has always been a major problem for data centers, leading to data loss. Current disk failure prediction approaches are mostly offline and assume that the disk labels required for training learning models are available and accurate. However, these offline methods are no longer suitable for disk failure prediction tasks in large-scale data centers. Behind this explosive amount of data, most methods do not consider whether it is not easy to get the label values during the training or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this article, Active Semi-supervised Learning Disk-failure Prediction ( ASLDP ), a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk lifecycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples with the most significant probability uncertainty and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP uses semi-supervised learning for pre-labeled by calculating the conditional values of the samples and enhances the generalization ability by active learning. Compared with several state-of-the-art offline and online learning approaches, the results on four realistic datasets from Backblaze and Baidu demonstrate that ASLDP achieves stable failure detection rates of 80–85% with low false alarm rates. In addition, we use a dataset from Alibaba to evaluate the generality of ASLDP . Furthermore, ASLDP can overcome the problem of missing sample labels and data redundancy in large data centers, which are not considered and implemented in all offline learning methods for disk failure prediction to the best of our knowledge. Finally, ASLDP can predict the disk failure 4.9 days in advance with lower overhead and latency.

Spae: Lifelong Disk Failure Prediction Via End-to-End Gan-Based Anomaly Detection with Ensemble Update

Spae: Lifelong Disk Failure Prediction Via End-to-End Gan-Based Anomaly Detection with Ensemble Update

Minority Disk Failure Prediction Based on Transfer Learning in Large Data Centers of Heterogeneous Disk Systems

The storage system in large scale data centers is typically built upon thousands or even millions of disks, where disk failures constantly happen. A disk failure could lead to serious data loss and thus system unavailability or even catastrophic consequences if the lost data cannot be recovered. While replication and erasure coding techniques have been widely deployed to guarantee storage availability and reliability, disk failure prediction is gaining popularity as it has the potential to prevent disk failures from occurring in the first place. Recent trends have turned toward applying machine learning approaches based on disk SMART attributes for disk failure predictions. However, traditional machine learning (ML) approaches require a large set of training data in order to deliver good predictive performance. In large-scale storage systems, new disks enter gradually to augment the storage capacity or to replace failed disks, leading storage systems to consist of small amounts of new disks from different vendors and/or different models from the same vendor as time goes on. We refer to this relatively small amount of disks as minority disks. Due to the lack of sufficient training data, traditional ML approaches fail to deliver satisfactory predictive performance in evolving storage systems which consist of heterogeneous minority disks. To address this challenge and improve the predictive performance for minority disks in large data centers, we propose a minority disk failure prediction model named TLDFP based on a transfer learning approach. Our evaluation results in two realistic datasets have demonstrated that TLDFP can deliver much more precise results and lower additional maintenance cost, compared to four popular prediction models based on traditional ML algorithms and two state-of-the-art transfer learning methods.

Cost‐efficiency disk failure prediction via threshold‐moving

SummarySelf‐Monitoring, Analysis, and Reporting Technology (SMART) is a technology in hard disk drives to predict impending disk failures for data repair in advance. As the prediction accuracy of SMART is unsatisfactory, recently, machine learning techniques have been explored to improve the prediction accuracy. Those approaches treat disk failure prediction as a binary classification problem and take SMART attributes as features, and some of them achieve satisfactory prediction accuracy. However, there is no uniform metric to measure the financial impact of these methods whose primary objective is to reduce disk failure recovery costs via disk failure prediction. In this article, from a financial impact perspective, we propose a simple, yet practical, metric Mean‐Cost‐To‐Recovery (MCTR) for disk failure prediction in data centers. Specifically, by assigning different weights to mispredicted healthy disks and failed disks, we measure the entire misprediction costs, that is, MCTR. In addition, we argue that the commonly used threshold 0.5 for disk failure prediction is suboptimal because of the fact of data imbalance, that is, failed disks are much fewer than healthy ones. To find the optimal threshold which renders minimal MCTR, we wrap a cost‐minimizing procedure around disk failure prediction and use a threshold‐moving technique for searching. Moreover, to map sample‐level prediction results to disk‐level prediction results, a modified leaky‐bucket algorithm is design to determine the disk health state by considering its multiple sample‐level prediction results. To evaluate the effectiveness of our approach, we conduct extensive experiments using three real‐world datasets. The experimental results show that compared with reactive data protection schemes, we can reduce MCTR by up to 86.9%, and compared with cost‐blind failure predictions, we can reduce MCTR by up to 22.3%.

A disk failure prediction method based on LSTM network due to its individual specificity

In current storage systems, to protect data security, disk failure prediction is required. Machine learning proved to be a method to solve the problem of disk failure prediction. However, because the disk-related values are affected by factors such as their use and usage environment, the values of different disks in the event of a failure are not the same. The normal value on one disk may be the value when another disk fails. Some studies have introduced the concept of time windows into disk failure prediction, trying to improve the ability of disk failure prediction by studying the relationship of a disk’s value change over time, and achieved good prediction results. We chose the neural network with time-series model to further validate the prediction of time series affect performance, and would like to be able to further improve the prediction performance by using a neural network. In this paper, we will introduce a disk failure prediction system based on LSTM networks. Considering the individual differences of the disks, we replace the input in the LSTM network with the continuous running records of the disks. The network will learn the disk information over a period of time and predict whether this disk will fail. With the proposed approach we are able to predict a disk will fail in next fifteen days with an average precision of 86.31. By comparing with other algorithms, our method performs well.

Incremental Prediction Model of Disk Failures Based on the Density Metric of Edge Samples

Disks are the main equipment for data storage in data centers. The prediction of disk failure is of great significance for the reliability and security of data. On account of the few abnormal samples in the disk datasets, it is difficult to satisfy the requirement of supervised and semi-supervised algorithms for the number of abnormal data while the unsupervised algorithms have poor performance on recall rate when solving the problems of local anomalies and wrapped a nomalies. This paper presents an incremental learning disk failure prediction model using the density metric of edge samples. An isolation region is built by searching the nearest neighbor of each sample. We calculate the nearest training point of the test point which is not a global anomaly and the nearest training point of the obtained nearest training point by Euclidean distance. The global metric of abnormal degree of the test sample comes from the ratio of the radius of the region where the two nearest training points are located. Then, the local metric of abnormal degree of the test sample comes from the ratio between the nearest distance from the test point to the edge of the training point region and the radius of the region. Abnormal scores of test points can be obtained by combining two measurements. We identify the SMART attributes that are significantly related to disk failures and promote their weights in the next time the attributes are inputted. The experiments are carried on the synthetic and public datasets which contain local anomalies and wrapped anomalies. The proposed method outperforms the typical unsupervised algorithms such as iNNE, iForest and LOF, and the achieved recall rates increase at most 7%. Furthermore, the contrast tests on the public disk datasets also verify the proposed method has better performance on recall rate.

New Metrics for Disk Failure Prediction That Go Beyond Prediction Accuracy

Prediction accuracy (true positives, false positives, and so on) is the usual way for evaluating disk-failure prediction models. Realistically however, we aim not only to correctly predict failures, but also to protect data against failure, i.e., we need to take appropriate action after a failure prediction. In the context of storage systems, protecting data requires that we migrate at-risk data, but this consumes network and disk bandwidth, which is particularly problematic for large-scale and cloud systems. This paper consolidates and builds on Li et al. (2016), where we propose using two new metrics, migration rate (MR) and mismigration rate (MMR), to measure the quality of disk failure prediction: MR measures how much at-risk data is migrated (and therefore protected) as a result of correct failure predictions, while MMR measures how much data is migrated needlessly as a result of incorrect failure predictions. In this paper, we additionally propose measuring quality in terms of migration time and mismigration time, which measure the time spent migrating at-risk disks, and the time spent mismigrating healthy disks caused by false alarms, respectively. To demonstrate these metrics’ usefulness, we use them to compare disk-failure prediction methods: we compare: 1) a classification tree (CT) model against a state-of-the-art recurrent neural network (RNN) model and 2) a gradient-boosted regression tree (GBRT) model (which predicts residual life) against RNN. We observe that while RNN performs best in the prediction accuracy experiments, the CT and GBRT models sometimes outperform RNN in the resource-dependent migration-rate experiments. We conclude that prediction accuracy is sometimes misleading: correct predictions do not necessarily imply protected data. We additionally present an improved GBRT model (GBRT+), which offers a practical improvement in disk residual-life prediction accordingly to the newly proposed metrics.

Disk failure prediction model for storage systems based on disk SMART technology

Recent years have seen remarkable advancements in storage systems designed to carry data. The data reliability issue has become increasingly important, as certain adverse factors can impact storage systems and cause data loss. In this study, we developed a disk failure prediction model using disk SMART technology for disk-based storage systems. The proposed model works in a twofold process: first, a parameter reliability model is established based on the actual threshold and attribute values of the SMART parameters, (which we determined through theoretical discussion, then experimentation and analysis) and second, a disk reliability model is built according to the parameter reliability model and parameter weights as determined. The disk reliability failure threshold is then determined by analyzing the SMART parameter data and parameter weights in tandem with the disk reliability model to fully determine the level of disk reliability. This approach eventually allows the user to migrate low-level reliability disks to backup disks, thereby improving the overall reliability of the storage system. Experimental results proved that the proposed approach is feasible and effective.

Fatman: Building Reliable Archival Storage Based on Low-Cost Volunteer Resources

We present Fatman, an enterprise-scale archival storage based on volunteer contribution resources from under-utilized web servers, usually deployed on thousands of nodes with spare storage capacity. Fatman is specifically designed for enhancing the utilization of existing storage resources and cutting down the hardware purchase cost. Two major concerned issues of the system design are maximizing the resource utilization of volunteer nodes without violating service level objectives (SLOs) and minimizing the cost without reducing the availability of archival system. Fatman has been widely deployed on tens of thousands of server nodes across several datacenters, providing more than 100 PB storage capacity and serving dozens of internal mass-data applications. The system realizes an efficient storage quota consolidation by strong isolation and budget limitation, to maximally support resource contribution without any degradation on host-level SLOs. It novelly improves data reliability by applying disk failure prediction to minish failure recovery cost, named fault-aware data management, dramatically reduces the mean time to repair (MTTR) by 76.3% and decreases file crash ratio by 35% on real-life product workload.

Fatman

We present Fatman, an enterprise-scale archival storage based on volunteer contribution resources from underutilized web servers, usually deployed on thousands of nodes with spare storage capacity. Fatman is specifically designed for enhancing the utilization of existing storage resources and cutting down the hardware purchase cost. Two major concerned issues of the system design are maximizing the resource utilization of volunteer nodes without violating Service Level Objectives (SLOs) and minimizing the cost without reducing the availability of archival system. Fatman has been widely deployed on tens of thousands of server nodes across several datacenters, provided more than 100PB storage capacity and served dozens of internal mass-data applications. The system realizes an efficient storage quota consolidation by strong isolation and budget limitation, to maximally support resources contribution without any degradation on host-level SLOs. It firstly improves data reliability by applying disk failure prediction to minish failure recovery cost, named fault-aware data management, dramatically reduces the MTTR by 76.3% and decreases file crash ratio by 35% on real-life product workload.