Multi-modal hybrid modeling strategy based on Gaussian Mixture Variational Autoencoder and spatial–temporal attention: Application to industrial process prediction

Abstract

The industrial process is characterized by its multi-modal nature and complex spatial and temporal correlations. Despite the fact that several multi-modal methods have been proposed, few of them can effectively extract deep multi-modal representations and the highly intricate spatial and temporal relationships. In this paper, a novel multi-modal hybrid modeling strategy (GMVAE-STA) is proposed for industrial process prediction. This strategy combines the Gaussian Mixture Variational Autoencoder (GMVAE) and the spatial–temporal attention based Gated Recurrent Unit (STA-GRU). First, the GMVAE maps the raw data to the latent space, which follows a Gaussian Mixture distribution, and the data with the highest probability in each Gaussian are identified as a mode. Then, the STA-GRU captures the complex spatial and temporal relationships within each mode and makes predictions. Experimental results on the Tennessee Eastman process and a real-world fluid catalytic cracking process demonstrate the effectiveness of mode classification and prediction of the proposed method.