HMT: Hybrid mechanistic Transformer for bio-fabrication prediction under complex environmental conditions

Abstract

In the field of bio-manufacturing, hybrid models that combine mechanistic models with data-driven models have not been widely adopted. This is primarily due to the complexity of the manufacturing process itself and the limited quality of available bio-manufacturing data. To address these limitations, this study introduces a novel hybrid model that combines a mechanistic-feature network with a Transformer-based data-driven model (HMT), effectively overcoming these challenges. The HMT integrates mechanistic and data-driven modeling to accurately predict target variables in bio-manufacturing processes. Initially, the HMT utilizes a mechanistic model to explore the feature relationships of the target sequence variables. It then compresses temporal domain variables into a univariate time series and applies spatial fusion weights for spatial domain variables, achieving variable fusion independently in each domain. Subsequently, a Transformer-based data-driven model is employed to model and predict the fused target variables, utilizing attention mechanisms to capture feature relationships within and between sequences. Upon comparison with various data-driven models, particularly those in the bio-manufacturing sector that employ artificial neural network approaches, the hybrid model demonstrates superior performance in mean squared error, mean absolute error, mean absolute percentage error, robustness, and Theil’s U statistic. The state-of-the-art performance of the HMT in long sequence prediction and short sequence prediction confirms its feasibility and potential for industrial applications.11The code is under the address: https://github.com/Scholar-Song/HMT.