BO-MADRSN: Bayesian optimized multi-attention residual shrinkage networks for industrial soft sensor modeling

Abstract

Convolutional neural networks have been widely utilized in industrial soft sensor modeling because of its excellent feature extraction capabilities in recent years. However, industrial data are often multimodal and contain noise, which affects the feature extraction capability of soft sensor models and leads to potentially large errors in the final key variable results. The solution to the above problems is based on denoising techniques and higher-level feature extraction techniques. Therefore, this paper proposes a Bayesian optimized Multi-Attention deep residual shrinkage network for industrial soft sensor model, referred to as BO-MADRSN. Firstly, in order to filter the noise in the original data during feature extraction, we used a soft thresholding method, where the thresholds are determined by back-propagation algorithm. Secondly, in order to retain the information of the data as much as possible and reduce the cases of zero after soft threshold filtering, we construct a residual network-like structure to complement the results obtained after soft threshold noise reduction by performing attentional feature extraction on the original features through the shortcut connections of this structure. Thirdly, we perform hierarchical denoising by stacking multiple residual blocks, and this structure facilitates network training. In addition, a Bayesian optimization strategy is introduced to select the optimal hyperparameters. In this paper, the results on several real industrial datasets verify the effectiveness and superiority of the proposed soft sensor model.