Convolutional Transformer: An Enhanced Attention Mechanism Architecture for Remaining Useful Life Estimation of Bearings

Abstract

Nowadays, deep learning (DL) methods for prognostic and health management (PHM) have vastly broadened the scope of applications in this field. Numerous approaches based on deep neural networks have been presented and applied to the remaining useful life (RUL) estimation of bearings. However, few of these methods are yet fully competent for the task of extracting degradation-related information from raw signals both locally and globally. To fill this research gap, we proposed a novel convolutional Transformer (CoT) which combines the global context capturing of attention mechanism with the local dependencies modeling of convolutional operation. Specifically, we designed a multi-scale convolutional (MSC) module with Swish activation for Transformer architecture to embed local feature learning into global sequence modeling. Our CoT fuses intra-token convolution and inter-token self-attention operations to enable simultaneous extraction of local dependencies and global interactions from the raw temporal signal into a trainable class token. Then, an end-to-end RUL estimation framework based on CoT is presented, which provides a mapping from raw vibration signals to estimated RULs. Finally, comprehensive case studies, including comparative studies and ablation experiments, fully validate the effectiveness and advancements of our CoT-based RUL estimation method.