Trans-Lighter: A light-weight federated learning-based architecture for Remaining Useful Lifetime prediction

Abstract

Predictive maintenance (PdM) plays an important role in industrial manufacturing. One of the most fundamental ideas underlying many PdM solutions is to estimate Remaining Useful Life (RUL) of machines. Recently, advanced deep learning models like convolutional neural network (CNN) and long short-term memory (LSTM) have been widely used for RUL prediction. However, these models also have certain limitations because of the difficulty in dealing with long-term dependencies in time series data. In this study, we propose a novel model based on transformer networks to overcome this difficulty. Rather than using the full structure of a transformer model, we exploit only the encoder combined with a linear layer. The Bayesian Optimization algorithm is applied to find optimal hyperparameters for the encoder. Experiments on widely used turbofan engine datasets show that our proposed method significantly outperforms the state-of-the-art RUL prediction methods by up to 25% in terms of predicting remaining usable life. We also provide a solution for the problem of preserving the privacy and security of data in smart manufacturing by designing a Federated Learning-based architecture for RUL using the proposed transformer-based model.