A weighted time embedding transformer network for remaining useful life prediction of rolling bearing

Abstract

Data-driven remaining useful life (RUL) prediction is of vital importance to industrial equipment prognostics health management (PHM). The transformer algorithms have been applied for RUL prediction recently. RUL prediction is a typically time-correlated task. However, in most transformer-based RUL prediction methods, one of the main procedures, patch embedding, is usually implemented by linear-based or convolution-based approaches, which lacks consideration for time correlation feature extraction. Moreover, the time correlation features at different degradation states may be variable, which will make different contributions to RUL prediction. To this end, in this paper, a novel framework for RUL prediction named weighted time embedding transformer (WTE-Trans) is proposed. A WTE module is designed to enhance the capacity of the model to extract more discriminative time correlation features and impose different weights automatically. Concretely, the peak-to-peak values are calculated from input and transformed firstly to provide prior knowledge information, and then multiplied by a learnable mask to generate an adaptive weight. Afterwards, the adaptive weight is used to impose constraints on the transformed input. Secondly, multiple 3D convolution (3D-conv) modules are designed to integrate the deep features through the time axis. Finally, the shifted-window transformer block is adopted to complete the RUL prediction task. Experiments are carried out on the widely used PHM 2012 bearing dataset and an actual industrial bearing full-lifecycle dataset to verify the effectiveness and application value. The results and analysis have shown that lower prediction error can be achieved by the proposed method.