Analysis of different RNN autoencoder variants for time series classification and machine prognostics

Abstract

Recurrent neural network (RNN) based autoencoders, trained in an unsupervised manner, have been widely used to generate fixed-dimensional vector representations or embeddings for varying length multivariate time series. These embeddings have been demonstrated to be useful for time series reconstruction, classification, and creation of health index (HI) curves of machines being used in industrial applications, based on which the remaining useful life (RUL) of machines can be estimated. In this study, we extend the traditional form of RNN autoencoders as a feature extractor for multivariate time series to a more general form in terms of arranging the order of input or output sequences and the hidden unit architecture. We apply the embeddings obtained by different variants of RNN autoencoders for a time series classification task and a machine RUL estimation problem using two publicly available datasets. A random research strategy is used to find the optimal hyperparameters of all variants for each task in order to give a fair comparison of the general performance among different variants over a large hyperparameter space, as well as the best performance that each variant can achieve compared with the best reported values in the literature. Our results show that traditional reversing the order of output time series while maintaining the order of input time series when training an RNN autoencoder does not show improved performance for the two studied cases. Thus, intentionally arranging the input or output order seems unnecessary for training the RNN autoencoder as a feature extractor of time series. We further observe that only the RNN architectures with gating mechanism can achieve the functionality of encoding for the time series, and none of the three common gated architectures we studied shows significantly and consistently improved performance compared to the others on the two studied cases. However, the bidirectional RNN autoencoders yield slightly better performance than their unidirectional counterparts.