Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction

Remaining Useful Life (RUL) prediction of rotating machinery plays a critical role in Prognostics and Health Management (PHM). Data-driven methods for RUL estimation have been widely developed because they don’t depend on much prior knowledge of the system. Recurrent neural network (RNN) is capable of modeling sequential data, which has been investigated for RUL prediction with statistical features of vibration signals in time domain and frequency domain. The drawback of utilizing statistical features is the ignorance of time-frequency information, which is critical in RUL prediction because the vibration signals are non-stationary when the fault occurs. To solve this problem, a novel deep architecture, named deep recurrent convolutional neural network (DRCNN) is proposed. By incorporating convolutional operation in the process of state transition of RNN, the spatial information in time-frequency domain can be automatically learned from the vibration signals, which contributes to the improvement of prediction performance. With convolutional operation in RNN, both spatial information in time-frequency domain and previous information are employed for RUL prediction. Furthermore, by stacking recurrent convolutional neural network layer by layer, the deep architecture can learn high-level features in the time-frequency domain. Finally, experimental analysis of RUL prediction using vibration signals of run-to-failure tests are carried out. Compared with the results of conventional deep RNN method, the proposed method shows its effectiveness and superiority.

Remaining useful life (RUL) prediction is an essential task of Prognostics and Health Management (PHM) of aircraft engines performed utilizing the huge data collected from multiple sensors attached to them to ensure their safe operation. While many studies have been reported on RUL prediction for aircraft engines, only a few of them focus on ensemble learning of CNN models for RUL prediction. The success of ensemble learning which is a combination of several base models developed using either same or different machine learning or deep learning algorithms, critically depends on the diversity among the base models generated. This paper proposes a new data-driven approach for RUL prediction of aircraft engines using ensemble learning based convolution neural networks (CNN) by investigating various steps to generate more diverse base models. The main objectives and contributions of this paper are as follows: Explore various CNN model architectures for RUL prediction - After data preprocessing and exploratory data analysis, two different CNN approaches, namely 2D CNN and 1D CNN with multiple channels, are investigated employing time window approach for time-series input preparation for better feature extraction by CNN. Each approach is experimented with multiple architectures to achieve the best possible outcome. Investigate engine specific RUL target function - For RUL prediction of turbofan engines using the C-MAPSS dataset, typically two RUL target functions, namely linear and piecewise linear, are used to determine RUL target values. In the piecewise linear RUL target function approach, which yields better performance in the reported studies, the RUL target value is assigned based on a piecewise linear degradation model which assumes a constant (and maximum) RUL target value in the early phase before linearly degrading the RUL targets. In the literature, this maximum RUL target value was chosen same for all the engines by taking a value of 125 or 130 without providing proper rationale. In this study, we adopt an approach based on the widely known health index to determine an engine specific initial (and maximum) RUL target value that can be used with the piecewise linear degradation model to determine RUL target values. Investigate hyperparameter optimization of CNN models to generate diverse base models - For the purpose of developing a high performance ensemble CNN model for RUL prediction, hyperparameter optimization of CNN models is performed to optimally determine the network structure (such as # of filters, filter size, stride, padding, # of convolutional, pooling, and dense layers, activation functions etc.) as well as the hyperparameters that determine the network training process (such as optimization method, learning rate, momentum, batch size etc.). Investigate ensemble learning to select and combine diverse CNN models for RUL prediction – In order to develop model combiners, diverse CNN models as base learners are selected using multiple performance measures such as RMSE, score function, MAE, and R2 score, and employing the non-negative least squares method, random forest regression, and extreme learning machine (ELM) to train model combiners. Evaluate the above proposed approach using the C-MAPSS dataset - To show the effectiveness of the proposed approach, various evaluations for RUL prediction using the popular C-MAPSS dataset (including all the four sub-datasets denoted as FD001, FD002, FD003 and FD004) are carried out and the results will be compared against the state-of-the-art results on the same dataset. Major emphasis of this proposed approach is on the generation of diverse CNN base models by carrying out various steps as explained above, and it is expected the results of this proposed approach will contribute towards enhancing the RUL prediction performance especially on the sub-datasets FD002 and FD004 which are challenging for the existing state-of-the-art RUL prediction techniques.

Remaining useful life prediction with uncertainty quantification based on multi-distribution fusion structure

As a novel paradigm in machine learning, deep transfer learning (DTL) can harness the strengths of deep learning for feature representation, while also capitalizing on the advantages of transfer learning for knowledge transfer. Hence, DTL can effectively enhance the robustness and applicability of the data-driven remaining useful life (RUL) prediction methods, and has garnered extensive development and research attention in machinery RUL prediction. Although there are numerous systematic review articles published on the topic of the DTL-based approaches, a comprehensive overview of the application of DTL in the RUL prediction for different mechanical equipment has yet to be systematically conducted. Therefore, it is imperative to further review the pertinent literature on DTL-based approaches. This will facilitate researchers in comprehending the latest technological advancements and devising efficient solutions to address the cross-domain RUL prediction challenge. In this review, a brief overview of the theoretical background of DTL and its application in RUL prediction tasks are provided at first. Then, a detailed discussion of the primary DTL methods and their recent advancements in cross-domain RUL prediction is presented. Next, the practical application of the current research is discussed in relation to the research object and its open-source data. More importantly, several challenges and further trend are further presented to conclude this paper in the end. We have reason to hope this work can offer convenience and inspiration to researchers seeking to advance in the field of RUL prediction.

Remaining Useful Life Prediction using Deep Learning Approaches: A Review

In recent years, prognostic and health management (PHM) has played an important role in industrial engineering. Efficient remaining useful life (RUL) prediction can ensure the development of maintenance strategies and reduce industrial losses. Recently, data-driven based deep learning RUL prediction methods have attracted more attention. The convolution neural network (CNN) is a kind of deep neural network widely used in RUL prediction. It shows great potential for application in RUL prediction. A CNN is used to extract the features of time-series data according to the spatial feature method. This way of processing features without considering the time dimension will affect the prediction accuracy of the model. On the contrary, the commonly used long short-term memory (LSTM) network considers the timing of the data. However, compared with CNN, it lacks spatial data extraction capabilities. This paper proposes a double-channel hybrid prediction model based on the CNN and a bidirectional LSTM network to avoid those drawbacks. The sliding time window is used for data preprocessing, and an improved piece-wise linear function is used for model validating. The prediction model is evaluated using the C-MAPSS dataset provided by NASA. The predicted results show the proposed prediction model to have a better prediction performance compared with other state-of-the-art models.

Due to less degradation data and the inconsistent data distribution of different bearings, remaining useful life (RUL) prediction methods based on deep learning still do not yield satisfactory predictive results. Using RUL prediction model trained with one bearing sample but tested with another bearing sample is challenging. To solve this problem, in this paper a new deep transfer learning-based RUL prediction method (DTL-RULPM) is proposed. We adopt min-max normalization to normalize the original vibration data of bearing. A three-layer sparse autoencoder is designed to extract the deep features of the source domain. Random data with standard normal distribution is generated with the consistent dimension of the high-dimensional features of the source domain. Maximum mean discrepancy (MMD) is used to minimize the probability distribution distance between the features of the source domain and the randomly generated data, such that the model can learn domain-invariant features of different bearings. Then we adopt a bi-directional long and short-term memory (Bi-LSTM) network to predict the RUL of the bearing. We use the IEEE PHM Challenge 2012 dataset to verify the proposed method. The results demonstrate that the proposed method improves the RUL prediction accuracy and robustness of different bearings.

A novel transfer gear life prediction method by the cross-condition health indicator and nested hierarchical binary-valued network

Journal bearings are the key components of the nuclear circulating water pump (NCWP), and accurate remaining useful life (RUL) prediction is of great significance for improving the reliability, safety, and maintenance planning of NCWP. However, it is difficult to quantify the uncertainty of bearing RUL based on the current deep learning (DL) model, resulting in a lack of credibility and effective convincing for RUL predicted by the model. Meanwhile, all existing hybrid models are basically simple combinations, and they cannot solve the uncertainty quantification problem of RUL predicted by DL. Hence, the bearing RUL prediction method based on a dynamic interactive hybrid model is proposed. Firstly, a degradation model based on a nonlinear enhanced generalized Wiener process (EGWP) is proposed, which combines gated neural networks and time-varying drift coefficients to describe the nonlinear degradation process of bearing. Then, a corrective gated recurrent unit (CGRU) network is designed to learn and predict real-time degradation increments, and the parameters of the degradation model are dynamically updated through the history and prediction of degradation increments. Finally, the bearing RUL prediction is given by the CGRU network, and the probability density function (PDF) of RUL is given by the proposed hybrid model. The performance of the proposed method is evaluated using the PHM 2012 bearing dataset and the NCWP journal bearing dataset. The results show that our proposed method can effectively predict bearing RUL and its uncertainty.

Bayesian large-kernel attention network for bearing remaining useful life prediction and uncertainty quantification

The study of intelligent operation and maintenance methods for turbofan engines is of great importance for improving the reliability of turbofan engines. Given the harsh operating conditions and complex structure of the turbofan engine, it is extremely difficult to establish an accurate physical model for remaining useful life (RUL) prediction. The traditional operation and maintenance method based on the physical model has several limitations in the application of turbofan engines, while the data-driven method offers a new solution. Compared with traditional machine learning models, deep learning models possess more powerful nonlinear expression capabilities and feature extraction capabilities. Therefore, this study focuses on studying the RUL prediction algorithm for turbofan engines based on the fused deep learning models. In this article, a multimodal deep learning approach based on a 1DCNN (1D convolutional neural network) + attention enhanced Bi-LSTM (bidirectional long short-term memory) network is proposed to predict the RUL by mining the temporal information of data. Furthermore, a DDResNet (dilated deep residual network) is also introduced to the 1DCNN submodel to leverage its hidden pattern mining capability due to its advance in preventing performance degradation across layers. Subsequently, the output of these two submodels is weighted fused to obtain the final RUL prediction. The merits of the proposed method are demonstrated by comparing it with existing methods for RUL prediction using the C-MAPSS (commercial modular aero-propulsion system simulation) dataset.

Remaining usefullife (RUL) prediction is crucial for simplifyingmaintenance procedures and extending the lifespan of aero-engines.Therefore, research on RUL prediction methods for aero-engines isincreasingly gaining attention. In particular, some existing deepneural networks based on multiscale features extraction have achievedcertain results in RUL predictions for aero-engines. However, thesemodels often overlook two critical factors that affect RUL predictionperformance: (i) different time series data points have varying importancefor RUL prediction, and (ii) the connections and similarities betweendifferent sensor data in both directions. This paper aims to extractvaluable multiscale features from raw monitoring data containing multiplesensor measurements, considering the aforementioned factors, and leveragethese features to enhance RUL prediction results. To this end, wepropose a novel deep neural network based on multiscale features extraction,named Multi-Scale Temporal-Spatial feature-based hybrid Deep neuralNetwork (MSTSDN). We conduct experiments using two aero-engine datasets, namely C-MAPSS and N-CMAPSS, to evaluate RUL prediction performanceof MSTSDN. Experimental results on C-MAPSS data set demonstrate thatMSTSDN achieves more accurate and timely RUL predictions comparedto 12 existing deep neural networks specifically designed for predictingRUL of aero-engine, especially under multiple operational conditionsand fault modes. And experimental results on N-CMAPSS data set eventuallyindicate that MSTSDN can effectively track and fit with the actualRUL during the engine degradation phase.

Remaining useful life (RUL) prediction is a cornerstone of Prognostic and Health Management (PHM), playing a crucial role in ensuring the reliability and safety of equipment. In recent years, deep learning techniques have shown great promise in RUL prediction, providing more reliable and accurate outcomes. However, many current deep learning models still fall short in comprehensive feature extraction. To tackle this limitation, we propose a multi-feature fusion model leveraging a dual-attention mechanism. Initially, convolutional neural networks (CNN) and channel attention mechanisms are employed to preliminarily extract spatial features. Subsequently, a layer combining Long Short-Term Memory (LSTM) and self-attention mechanisms is used to further extract and integrate temporal features. Concurrently, domain knowledge-based handcrafted features are selected for multi-feature fusion. Finally, RUL values are predicted via regression. The effectiveness of the proposed method is validated on widely-used academic datasets, and its superior performance in RUL prediction is demonstrated through comparative analysis with other methods.

A Health state-related ensemble deep learning method for aircraft engine remaining useful life prediction

Accurate Remaining Useful Life (RUL) prediction for complex systems like aero-engines is vital for optimizing maintenance strategies. However, the intrinsic complexity of engine degradation, marked by highly coupled sensor dependencies and multi-scale temporal dynamics, continues to pose a fundamental challenge to data-driven approaches. Existing methods often exhibit incomplete temporal feature modeling and fail to capture the intrinsic spatial coupling among multi-channel sensor signals. Moreover, domain shifts caused by fluctuating flight conditions and hybrid fault modes lead to significant performance degradation in unlabeled target domains.To address these challenges, this paper proposes a deep adversarial domain adaptive transfer learning model named GMSIT-WDADA.First, we design GMSIT, a feature extractor that incorporates gated convolution, a selective state space model, and a sparse inverted Transformer. This architecture achieves the extraction of multi-scale temporal features, including short-term transient local features, long-term trend global features, and cross-temporal correlations, while explicitly capturing the spatial coupling relationships across sensors.Second, a neural network is employed to estimate the Wasserstein distance as a replacement for the conventional domain discriminator, achieving more stable cross-domain feature alignment.Experimental results on the C-MAPSS dataset demonstrate that the proposed model can accurately predict RUL values. Furthermore, RUL prediction experiments on 12 cross-domain scenarios of turbofan engines demonstrate that GMSIT-WDADA performs reliable RUL prediction in unlabeled target domains, achieving 7% lower average RMSE and 39% lower average Score compared with the best methods, further verifying the superiority and robustness of the proposed method.