Benchmarking Transfer Learning Strategies in Time-Series Imaging: Recommendations for Analyzing Raw Sensor Data

Abstract

With the growing availability and complexity of time-series sequences, scalable and robust machine learning approaches are required that overcome the sampling challenge of quantitatively sufficient training data. Following the research trend towards the deep learning-based analysis of time-series encoded as images, this study proposes a time-series imaging workflow that overcomes the challenge of quantitatively limited sensor data across domains (i.e., medicine and engineering). After systematically identifying the three relevant dimensions that affect the performance of the deep learning-based analysis of visualized time-series data, we performed a benchmarking evaluation with a total of 24 unique convolutional neural network models. Following a two-level transfer learning investigation, we reveal that fine-tuning the mid-level features results in the best classification performance. As a result, we present an optimized representation of the VGG16 network, which outperforms previous studies in the field. Our approach is accurate, robust, and manifests internal and external validity. By only using the raw time-series data, our model does not require manual feature engineering, being of high practical relevance. As the post-hoc analysis of our results reveals that our model allows automated extraction of meaningful features based on the trend of the underlying time-series data, our study also adds to explainable artificial intelligence. Furthermore, our proposed workflow reduces the sequence length of the input data while preserving all information. Especially with the hurdle of long-term dependencies in sequential time-series data, we overcome related work’s limitation of the vanishing gradients problem and contribute to the sequential learning theory in artificial intelligence.