Multi-scale self-supervised representation learning with temporal alignment for multi-rate time series modeling

Abstract

Deep sequential networks have shown great power in time series regression and classification. So far, most approaches naturally assume that the time sequential data are uniformly sampled. In practice, however, different variables usually have different sampling rates, thereby forming multi-rate time series (MR-TS). Particularly, the target variable (i.e., label) to be predicted usually has a lower sampling frequency due to the difficulty of manual annotations. The multi-rate problem poses two challenges. One is the diverse dynamics at different sampling rates, which is defined as multi-scale dynamics. The other is label scarcity. To tackle the above obstacles, this paper developed a Multi-scale Self-supervised Representation Learning technique with a Temporal Alignment mechanism (MSRL-TA) as a coherent framework. Concretely, a probabilistic masked autoencoding approach is pertinently developed, in which segment-wise masking schemes and rate-aware positional encodings are devised to enable the characterization of multi-scale temporal dynamics. In the course of pre-training, the encoder networks are able to generate rich and holistic representations of multi-rate data, thereby alleviating the label scarcity issue for supervised fine-tuning. Furthermore, a Temporal Alignment mechanism is devised to refine synthesized features for dynamic predictive modeling through feature block division and block-wise convolution. With empirical evaluation through extensive experiments, our proposed MSRL-TA achieved consistent state-of-the-art in both multi-rate time series regression and classification tasks on five real-world datasets, including air quality prediction, industrial soft sensing, human activity recognition, and speech voice classification.