Time Series Classification Tasks Research Articles

An Efficient Federated Distillation Learning System for Multitask Time Series Classification

This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components: a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. For each user, the FBST framework transfers knowledge from its teacher’s hidden layers to its student’s hidden layers via knowledge distillation, where the teacher and student have identical network structures. For each connected user, its student model’s hidden layers’ weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user’s and its partner’s weights are the closest among all the weights uploaded. The server exchanges and sends back the user’s and its partner’s weights to these two users which then load the received weights to their teachers’ hidden layers. Experimental results show that compared with a number of state-of-the-art federated learning algorithms, our proposed EFDLS wins 20 out of 44 standard UCR2018 datasets and achieves the highest mean accuracy (70.14%) on these datasets. In particular, compared with a single-task Baseline, EFDLS obtains 32/4/8 regarding ’win’/’tie’/’lose’ and results in an improvement of approximately 4% in terms of mean accuracy.

Main requirements of end-to-end deep learning models for biomedical time series classification in healthcare environments

The use of deep learning models with medical data is becoming more widespread. However, although numerous models have shown high accuracy in medical-related tasks, such as medical image recognition (e.g. radiographs), there are still many problems with seeing these models operating in a real healthcare environment. This article presents a series of basic requirements that must be taken into account when developing deep learning models for biomedical time series classification tasks, with the aim of facilitating the subsequent production of the models in healthcare. These requirements range from the correct collection of data, to the existing techniques for a correct explanation of the results obtained by the models. This is due to the fact that one of the main reasons why the use of deep learning models is not more widespread in healthcare settings is their lack of clarity when it comes to explaining decision making.

A Framework for Ultra Low-Power Hardware Accelerators Using NNs for Embedded Time Series Classification

In embedded applications that use neural networks (NNs) for classification tasks, it is important to not only minimize the power consumption of the NN calculation, but of the whole system. Optimization approaches for individual parts exist, such as quantization of the NN or analog calculation of arithmetic operations. However, there is no holistic approach for a complete embedded system design that is generic enough in the design process to be used for different applications, but specific in the hardware implementation to waste no energy for a given application. Therefore, we present a novel framework that allows an end-to-end ASIC implementation of a low-power hardware for time series classification using NNs. This includes a neural architecture search (NAS), which optimizes the NN configuration for accuracy and energy efficiency at the same time. This optimization targets a custom designed hardware architecture that is derived from the key properties of time series classification tasks. Additionally, a hardware generation tool is used that creates a complete system from the definition of the NN. This system uses local multi-level RRAM memory as weight and bias storage to avoid external memory access. Exploiting the non-volatility of these devices, such a system can use a power-down mode to save significant energy during the data acquisition process. Detection of atrial fibrillation (AFib) in electrocardiogram (ECG) data is used as an example for evaluation of the framework. It is shown that a reduction of more than 95% of the energy consumption compared to state-of-the-art solutions is achieved.

TCRAN: Multivariate time series classification using residual channel attention networks with time correction

Currently, the most popular and effective approach to solve multivariate time series classification(MTSC) tasks is based on deep learning technology. However, the existing deep learning-based algorithms ignore the unique time characteristics of time series in the process of network training, and do not consider the features correlation in different convolutional layers. So they cannot obtain the convincing feature representation ability and result in unsatisfactory classification accuracy. To solve this problem, we propose a new time corrected residual attention network(TCRAN) which can fully extract the long-term time-dependence information to enhance the discriminative power of the network. The hallmark of TCRAN is that we employ the time residual channel attention block(TRCAB) as the basic structure, which incorporates the adaptive channel feature adjustment mechanism(AFM) and the bi-directional gated recurrent unit(Bi-GRU) into the deep residual structure to adaptively extract time-dependent features. Meanwhile, to integrate the overall dependency information between different layers, we also employ an inter-module adaptive feature adjustment mechanism(IAM) in the TCRAN. The experiments results with 15 multivariate time series datasets illustrate that the proposed TCRAN can achieve the highest average classification accuracy of 0.7276 and improve accuracy by 1.64% compared to the state-of-the-art method. All these verify the effectiveness of TCRAN.

Random Subspace Ensembles of Fully Convolutional Network for Time Series Classification

Time series classification (TSC) task is one of the most significant topics in data mining. Among all methods for this issue, the deep-learning-based shows superior performance for its good adaption to raw series data and automatic extraction of features. However, rare eyes are kept on composing ensembles of these superior individual classifiers to achieve further breakthroughs. The existing deep learning ensembles NNE did a heavy work of combining 60 individuals but did not maximize the deserving improvement, since it merely pays attention to the diversity of individuals but ignores their accuracy. In this paper, we propose to construct an ensemble of Full Convolutional Neural Networks (FCN) by Random Subspace Method (RSM), named RSM-FCN. FCN is a simple but outstanding individual classifier and RSM is suitable for high dimensional data such as time series, but there are few instances. Thus, the combination of these strengths, RSM-FCN provides a highly cost-effective approach to yield promising results. Experiments on the UCR dataset demonstrate the effectiveness and reasonability of the proposed method.

An Improved Self-Training Method for Positive Unlabeled Time Series Classification Using DTW Barycenter Averaging.

Traditional supervised time series classification (TSC) tasks assume that all training data are labeled. However, in practice, manually labelling all unlabeled data could be very time-consuming and often requires the participation of skilled domain experts. In this paper, we concern with the positive unlabeled time series classification problem (PUTSC), which refers to automatically labelling the large unlabeled set U based on a small positive labeled set PL. The self-training (ST) is the most widely used method for solving the PUTSC problem and has attracted increased attention due to its simplicity and effectiveness. The existing ST methods simply employ the one-nearest-neighbor (1NN) formula to determine which unlabeled time-series should be labeled. Nevertheless, we note that the 1NN formula might not be optimal for PUTSC tasks because it may be sensitive to the initial labeled data located near the boundary between the positive and negative classes. To overcome this issue, in this paper we propose an exploratory methodology called ST-average. Unlike conventional ST-based approaches, ST-average utilizes the average sequence calculated by DTW barycenter averaging technique to label the data. Compared with any individuals in PL set, the average sequence is more representative. Our proposal is insensitive to the initial labeled data and is more reliable than existing ST-based methods. Besides, we demonstrate that ST-average can naturally be implemented along with many existing techniques used in original ST. Experimental results on public datasets show that ST-average performs better than related popular methods.

LSTM-MFCN: A time series classifier based on multi-scale spatial–temporal features

Time series classification (TSC) task attracts huge interests, since they correspond to the real-world problems in a wide variety of fields, such as industry monitoring. Deep learning methods, especially CNN and FCN, shows competitive performance in TSC task by their virtue of good adaption for raw time series and self-adapting extraction of features. Then various variants of CNN are proposed so as to make further breakthrough by the better perception to characteristics of data. Among them, LSTM-FCN and GRU-FCN who learn spatial and temporal features simultaneously are the most remarkable ones, achieving state of the art results. Therefore, inspired by their success and in consideration of the discriminative features implied in time series are diverse in size, a multimodal network LSTM-MFCN composed of multi-scale FCN (MFCN) and LSTM are proposed in this work. The gate-based network LSTM naturally fits to various terms time dependencies, and FCN with multi-scale sets of filters are capable to perceive spatial features of different range from time series curves. Besides, dilation convolution is deployed to build multi-scale receptive fields in larger level without increasing the parameters to be trained. The full perception of large multi-scale spatial–temporal features lead LSTM-MFCN to possess comprehensive and thorough grasp to time series, thus achieve even better accuracies. Finally, two representative architectures are presented specifically and their experiments on UCR datasets reveals the effectiveness and superiority of proposed LSTM-MFCN.

Machine Learning on FPGA for Robust ${\rm {Si}}_{3}{\rm {N}}_{4}$-Gate ISFET pH Sensor in Industrial IoT Applications

This article presents performance enhancement of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\rm {Si}}_{3}{\rm {N}}_{4}$</tex-math></inline-formula> -gate ion-sensitive field-effect transistor based pH sensor using machine learning (ML) techniques. A robust SPICE macromodel is developed using experimental data, which incorporates intrinsic temperature and temporal characteristics of the device, which is further used in sensor readout circuit (ROIC), which shows a nonideal temperature and time dependence in the voltage output. To make the device robust to the critical drifts, we exploit six state-of-the-art ML models, which are trained using the data generated from ROIC for a wide range of pH, temperature, and temporal conditions. Thorough comparison between ML models shows random forest outperforms other models for drift compensation task. This work also shows a preliminary time series classification task. The ML models are implemented on a Xilinx PYNQ-Z1 field-programmable gate array (FPGA) board to validate the performance in power and memory-restricted environment, crucial for IoT applications. A parameter, implementation factor is defined to evaluate best ML model for IoT deployment using FPGA/MCU hardware implementation. The significantly lower power consumption of FPGA board as compared to CPU with no noticeable performance drop is a pointer to the future of robust pH sensors used in industrial and remote IoT applications.

Time series classification using diversified Ensemble Deep Random Vector Functional Link and Resnet features

Random Vector Functional Link (RVFL) is popular among researchers in many areas of machine learning. RVFL is preferred by many researchers as RVFL can produce good performance with relatively little training time. Recent works extend RVFL into deep and ensemble versions. However, RVFL does not have effective feature extraction methods commonly used in time series classification. This results in poor performance of RVFL in time series classification tasks. Also, deep RVFL is a relatively new and evolving area of research. In this paper, we present a framework that extracts features from Residual Networks (Resnet) and trains Ensemble Deep Random Vector Functional Link (edRVFL). We use features extracted from every residual block to train an ensemble of edRVFLs. We propose the following enhancements to edRVFL. Firstly, we diversity the structure of edRVFL and the direct link features to encourage diversity. Secondly, we built an ensemble of edRVFLs with the top two activation functions. Thirdly, we use two-stage tuning to save computational costs. Lastly, we perform a weighted average of all decisions made by every edRVFL. Experiments on the 55 largest UCR datasets show that using features extracted from all Residual blocks improves performance. All our proposed enhancements help improve classification accuracy or computational effort. Consequently, our proposed framework outperforms all traditional and deep learning-based time series classification methods.

Evaluation of interpretability methods for multivariate time series forecasting.

Being able to interpret a model’s predictions is a crucial task in many machine learning applications. Specifically, local interpretability is important in determining why a model makes particular predictions. Despite the recent focus on interpretable Artificial Intelligence (AI), there have been few studies on local interpretability methods for time series forecasting, while existing approaches mainly focus on time series classification tasks. In this study, we propose two novel evaluation metrics for time series forecasting: Area Over the Perturbation Curve for Regression and Ablation Percentage Threshold. These two metrics can measure the local fidelity of local explanation methods. We extend the theoretical foundation to collect experimental results on four popular datasets. Both metrics enable a comprehensive comparison of numerous local explanation methods, and an intuitive approach to interpret model predictions. Lastly, we provide heuristical reasoning for this analysis through an extensive numerical study.

CONVOLUTIONAL NEURAL NETWORKS FOR DETECTING BRIDGE CROSSING EVENTS WITH GROUND-BASED INTERFEROMETRIC RADAR DATA

Abstract. This study focuses on detecting vehicle crossings (events) with ground-based interferometric radar (GBR) time series data recorded at bridges in the course of critical infrastructure monitoring. To address the challenging event detection and time series classification task, we rely on a deep learning (DL) architecture. The GBR-displacement data originates from real-world measurements at two German bridges under normal traffic conditions. As preprocessing, we only apply a low-pass filter. We develop and evaluate a one-dimensional convolutional neural network (CNN) to achieve a solely data-driven event detection. As a baseline machine learning approach, we use a Random Forest (RF) with a selected feature-based input. Both models’ performance is evaluated on two datasets by focusing on identifying events and pure bridge oscillations. Generally, the event classification results are promising, and the CNN outperforms the RF with an overall accuracy of 94.7% on the test subset. By relying on an entirely unknown second dataset, we focus on the models’ performances regarding the distinction between events and decays. On this dataset, the CNN meets this challenge successfully, while the feature-based RF classifies the majority of non-event decays as events. To sum up, the presented results reveal the potential of a data-driven DL approach concerning the detection of bridge crossing events in GBR-based displacement time series data. Based on such an event detection, a prospective assessment of bridge conditions seems feasible as an extension to previous structural health monitoring approaches.

Echo state network with a global reversible autoencoder for time series classification

• In order to reestablish the random initialization input weights of ESN, a single layer ESN with a global reversible AE (GRAE) algorithm is proposed to ensure that the outputs of GRAE are remarkably correlated with the input data. • The reservoir layer with a reversible function is calculated by pulling the ESN output back and injecting it into the reservoir layer. Thus, the feature learning is enriched by additional information, which results in a better performance. • The GRAE which is based on ridge regression to solve a least squares problem is entirely different from existing back-propagation (BP) based AEs. So, the training speed of GRAE is many times faster than that of the BP based AEs. • In existing ESN autoencoder methods, all weights in the reservoir layer are generated randomly. The simple cycle connection is utilized in the reservoir layer of ESN to make sure the reservoir weights can be generated deterministically. An echo state network (z) can provide an efficient dynamic solution for predicting time series problems. However, in most cases, ESN models are applied for predictions rather than classifications. The applications of ESN in time series classification (TSC) problems have yet to be fully studied. Moreover, the conventional randomly generated ESN is unlikely to be optimal because of the randomly generated input and reservoir weights, which are not always guaranteed to be optimal. Randomly generating all layer weights is improper, because a purely random layer might destroy the useful features. To overcome this disadvantage, this study provides a new input weight establishment framework of ESN based on autoencoder (AE) theory for TSC tasks. A global reversible AE (GRAE) algorithm is proposed to reestablish the random initialization input weights of the ESN. In existing ESN-AEs, the output weights obtained in the encoding process are directly reused as the initial input weights. By contrast, in GRAE, the reservoir layer with a reversible activation function is calculated by pulling the decoding layer output back and injecting it into the reservoir layer. Thus, feature learning is enriched by additional information, which results in improved performance. The current weights of the encoding layer are iteratively replaced by the decoding layer to ensure that the outputs of the GRAE are remarkably correlated with the input data. Visualization analyses and experiments of the input weights on a massive set of UCR time series datasets indicate that the proposed GRAE method can considerably improve the original two-layer ESN-based classifiers and the proposed GRAE-ESN classifier yields better performance compared with traditional state-of-the-art TSC classifiers. Furthermore, the proposed method can provide comparable performance and considerably faster training speed compared with three deep learning classifiers.

A novel neural network based on dynamic time warping and Kalman filter for real-time monitoring of supersonic inlet flow patterns

A supersonic inlet is one of the key components in a supersonic air-breathing propulsion system and is the basis for protection control. The overall system performance can be greatly influenced by its flow patterns, so it plays a crucial part and is necessary to develop methods for monitoring its flow patterns to ensure stable and safe operation. This issue can be viewed as a time series classification (TSC) task. Traditionally, several manually-engineered features are extracted as the indicators to evaluate the operation status, but this process can be heavily dependent on the professional experience. In this paper, a novel neural network called DTW-SLFN-KF is proposed, which integrates Dynamic Time Warping (DTW) and Kalman Filter (KF) into a single-hidden-layer neural network (SLFN) architecture to directly determine flow patterns from the dynamic sensor signals. The proposed network first adopts a DTW layer as the feature extractor to automatically extract robust features, and exploits the flexible alignment ability of DTW to keep the temporal continuity and deal with the temporal distortions. Then, these features are fed into an SLFN for classification. After that, to make full use of the extracted features and improve the classification performance of SLFN when the network structure is fixed, KF is applied as a linear post-processing technique to get the predicted output of SLFN closer to the true output. Experimental results demonstrate that the proposed DTW-SLFN-KF network has better comprehensive performance for monitoring the flow patterns of supersonic inlet in terms of monitoring accuracy and real-time performance when compared with other competitive methods.

Mo-BoNet: A TIME SERIES CLASSIFICATION MODEL BASED ON COMPUTER VISION

Time series are widely distributed in many fields. Classical statistical methods are difficult to model the deep meaning of time series, and the deep learning methods based on recurrent neural network has great limitations when it is applied to indefinite long time series. In order to solve the above problems, a time series classification model based on computer vision is proposed, which transforms the time series classification problem into image classification problem. Firstly, three kinds of images with different linewidth corresponding to the time series are used as input to reduce the information loss in the conversion process. Secondly, the transfer learning model based on MobileNetV3-Large is used to encode the image data, and XGBoost is used for classification. The experimental results show that the classification effect of this model is better than that of the classical image classification model, and its XGBoost is also better than other ensemble methods, which proves the feasibility of computer vision method in time series classification task.

Robust video content analysis schemes for human action recognition.

Action recognition is a challenging time series classification task that has received much attention in the recent past due to its importance in critical applications, such as surveillance, visual behavior study, topic discovery, security, and content retrieval. The main objective of the research is to develop a robust and high-performance human action recognition techniques. A combination of local and holistic feature extraction methods used through analyzing the most effective features to extract to reach the objective, followed by using simple and high-performance machine learning algorithms. This paper presents three robust action recognition techniques based on a series of image analysis methods to detect activities in different scenes. The general scheme architecture consists of shot boundary detection, shot frame rate re-sampling, and compact feature vector extraction. This process is achieved by emphasizing variations and extracting strong patterns in feature vectors before classification. The proposed schemes are tested on datasets with cluttered backgrounds, low- or high-resolution videos, different viewpoints, and different camera motion conditions, namely, the Hollywood-2, KTH, UCF11 (YouTube actions), and Weizmann datasets. The proposed schemes resulted in highly accurate video analysis results compared to those of other works based on four widely used datasets. The First, Second, and Third Schemes provides recognition accuracies of 57.8%, 73.6%, and 52.0% on Hollywood2, 94.5%, 97.0%, and 59.3% on KTH, 94.5%, 95.6%, and 94.2% on UCF11, and 98.9%, 97.8% and 100% on Weizmann. Each of the proposed schemes provides high recognition accuracy compared to other state-of-art methods. Especially, the Second Scheme as it gives excellent comparable results to other benchmarked approaches.

An ensemble radius basis function network based on dynamic time warping for real-time monitoring of supersonic inlet flow patterns

As the basis of protection control, supersonic inlet plays an important role in a supersonic air-breathing propulsion system, so it is of great significance to ensure the safe and stable operation by monitoring its flow patterns. From the perspective of machine learning, this issue can be viewed as a time series classification (TSC) task. Traditionally, several manually-engineered features are extracted as the indicators to evaluate the operation status, which can be heavily dependent on the professional experience and time-consuming. This paper proposes a novel Dynamic Time Warping-Radius Basis Function (DTW-RBF) network to directly determine the flow patterns from the dynamic input signals. DTW-RBF network replaces the Euclidean distance in the static RBF kernels with the DTW distance, which exploits the elastic matching ability of DTW to align the input signals to the kernels. Then, the second-order Levenberg-Marquarelt (LM) optimization algorithm is used to allow the efficient training process of the proposed network. In order to determine the appropriate locations for sensor placement and enhance the robustness and reliability of monitoring flow patterns by a single sensor, an optimal subset of sensors is further selected for ensemble through multi-objective optimization and fuzzy decision. Experimental results demonstrate that the proposed DTW-RBF network works efficiently for TSC tasks on benchmark time series datasets, and has better comprehensive performance for monitoring supersonic inlet flow patterns in terms of classification accuracy and test time. The ensemble classifier further increases the classification accuracy with still meeting the real-time requirements.

On exploring feature representation learning of items to forecast their rise and fall in social media

User-item interactions in social media provide a rich dataset for wide applications such as viral marketing and recommender systems. Post retweeting behaviors and venue check-in events by users are the most representative. While existing studies predict items’ rise and fall, i.e., tweet popularity and venue closure detection, using hand-crafted features, this paper aims at exploring feature representation learning to improve prediction performance. We target at two essential time-series classification tasks on social media, including Shutdown Risk Prediction (SRP) of venues and Tweet Popularity Prediction (TPP) of posts. We study how feature representation learning of items can benefit both SRP and TPP tasks. The main idea is to learn item embedding vectors as features in item-item graphs constructed from time series of check-in events and retweeting behaviors. The learned features are used together with manually-defined features to enlarge the representation capability. In the TPP task, we also propose a pattern-aware self-exciting point process (PSEISMIC) model to generate time-series features. Experiments conducted on Instagram, Foursquare, and Twitter datasets exhibit promising performance of jointly utilizing learned and extracted features in both tasks. PSEISMIC can also further boost TPP accuracy. The major contribution of this work is three-fold. First, we propose to jointly deal with SRP and TPP under the same framework of feature extraction and learning. Second, we show that feature presentation learning of items can benefit these two prediction tasks with time series data. Third, by incorporating time series patterns, the proposed PSEISMIC further improves the performance of popularity prediction.

Risk Prediction for Ship Encounter Situation Awareness Using Long Short-Term Memory Based Deep Learning on Intership Behaviors

Encounter risk prediction is critical for safe ship navigation, especially in congested waters, where ships sail very near to each other during various encounter situations. Prior studies on the risk of ship collisions were unable to address the uncertainty of the encounter process when ignoring the complex motions constituting the dynamic ship encounter behavior, which may seriously affect the risk prediction performance. To fill this gap, a novel AIS data-driven approach is proposed for ship encounter risk prediction by modeling intership behavior patterns. In particular, multidimensional features of intership behaviors are extracted from the AIS trace data to capture spatial dependencies between encountering ships. Then, the challenging task of risk prediction is to discover the complex and uncertain relationship between intership behaviors and future collision risk. To address this issue, we propose a deep learning framework. To represent the temporal dynamics of the encounter process, we use the sliding window technique to generate the sequences of behavioral features. The collision risk level at a future time is taken as the class label of the sequence. Then, the long short-term memory network, which has a strong ability to model temporal dependency and complex patterns, is extended to establish the relationship. The benefit of our approach is that it transforms the complex problem for risk prediction into a time series classification task, which makes collision risk prediction reliable and easier to implement. Experiments were conducted on a set of naturalistic data from various encounter scenarios in the South Channel of the Yangtze River Estuary. The results show that the proposed data-driven approach can predict future collision risk with high accuracy and efficiency. The approach is expected to be applied for the early prediction of encountering ships and as decision support to improve navigation safety.

Performance Boosting of Scale and Rotation Invariant Human Activity Recognition (HAR) with LSTM Networks Using Low Dimensional 3D Posture Data in Egocentric Coordinates

Human activity recognition (HAR) has been an active area in computer vision with a broad range of applications, such as education, security surveillance, and healthcare. HAR is a general time series classification problem. LSTMs are widely used for time series classification tasks. However, they work well with high-dimensional feature vectors, which reduce the processing speed of LSTM in real-time applications. Therefore, dimension reduction is required to create low-dimensional feature space. As it is experimented in previous study, LSTM with dimension reduction yielded the worst performance among other classifiers, which are not deep learning methods. Therefore, in this paper, a novel scale and rotation invariant human activity recognition system, which can also work in low dimensional feature space is presented. For this purpose, Kinect depth sensor is employed to obtain skeleton joints. Since angles are used, proposed system is already scale invariant. In order to provide rotation invariance, body relative direction in egocentric coordinates is calculated. The 3D vector between right hip and left hip is used to get the horizontal axis and its cross product with the vertical axis of global coordinate system assumed to be the depth axis of the proposed local coordinate system. Instead of using 3D joint angles, 8 number of limbs and their corresponding 3D angles with X, Y, and Z axes of the proposed coordinate system are compressed with several dimension reduction methods such as averaging filter, Haar wavelet transform (HWT), and discrete cosine transform (DCT) and employed as the feature vector. Finally, extracted features are trained and tested with LSTM (long short-term memory) network, which is an artificial recurrent neural network (RNN) architecture. Experimental and benchmarking results indicate that proposed framework boosts the performance of LSTM by approximately 30% accuracy in low-dimensional feature space.

Analysis of different RNN autoencoder variants for time series classification and machine prognostics

Recurrent neural network (RNN) based autoencoders, trained in an unsupervised manner, have been widely used to generate fixed-dimensional vector representations or embeddings for varying length multivariate time series. These embeddings have been demonstrated to be useful for time series reconstruction, classification, and creation of health index (HI) curves of machines being used in industrial applications, based on which the remaining useful life (RUL) of machines can be estimated. In this study, we extend the traditional form of RNN autoencoders as a feature extractor for multivariate time series to a more general form in terms of arranging the order of input or output sequences and the hidden unit architecture. We apply the embeddings obtained by different variants of RNN autoencoders for a time series classification task and a machine RUL estimation problem using two publicly available datasets. A random research strategy is used to find the optimal hyperparameters of all variants for each task in order to give a fair comparison of the general performance among different variants over a large hyperparameter space, as well as the best performance that each variant can achieve compared with the best reported values in the literature. Our results show that traditional reversing the order of output time series while maintaining the order of input time series when training an RNN autoencoder does not show improved performance for the two studied cases. Thus, intentionally arranging the input or output order seems unnecessary for training the RNN autoencoder as a feature extractor of time series. We further observe that only the RNN architectures with gating mechanism can achieve the functionality of encoding for the time series, and none of the three common gated architectures we studied shows significantly and consistently improved performance compared to the others on the two studied cases. However, the bidirectional RNN autoencoders yield slightly better performance than their unidirectional counterparts.