Hierarchical Self-Attention Network for Industrial Data Series Modeling With Different Sampling Rates Between the Input and Output Sequences.

Abstract

For industrial processes, it is significant to carry out the dynamic modeling of data series for quality prediction. However, there are often different sampling rates between the input and output sequences. For the most traditional data series models, they have to carefully select the labeled sample sequence to build the dynamic prediction model, while the massive unlabeled input sequences between labeled samples are directly discarded. Moreover, the interactions of the variables and samples are usually not fully considered for quality prediction at each labeled step. To handle these problems, a hierarchical self-attention network (HSAN) is designed for adaptive dynamic modeling. In HSAN, a dynamic data augmentation is first designed for each labeled step to include the unlabeled input sequences. Then, a self-attention layer of variable level is proposed to learn the variable interactions and short-interval temporal dependencies. After that, a self-attention layer of sample level is further developed to model the long-interval temporal dependencies. Finally, a long short-term memory network (LSTM) network is constructed to model the new sequence that contains abundant interactions for quality prediction. The experiment on an industrial hydrocracking process shows the effectiveness of HSAN.