Novel deep-learning method based on LSA-Transformer for fault detection and its implementation in penicillin fermentation process

Abstract

Transformer models have proved to be excellent in solving long-distance dependency problems for many tasks. However, a large amount of computation is required to achieve a high performance. In addition, this model has certain limitations in its ability to extract local information. In this paper, a local information enhancement and sparse attention mechanism Transformer (LSA-Transformer) model is proposed to address these issues. First, local information between data is captured from deep and multiple scales to achieve feature fusion and enhancement of local information. Second, by the sparse attention mechanism, the long-distance dependency relationship of the data is preserved. Third, the computational complexity of the improved model is reduced from quadratic to linear, while retaining the ability to capture long-distance dependencies. Finally, experiments in the penicillin fermentation process show that the improved Transformer model achieves significant improvements compared to existing methods.