Constructing Time-varying and History-dependent Kinetic Models Via Reinforcement Learning

Abstract

In Chapter 8, we demonstrated how to identify an accurate lumped kinetic model structure through reaction network reduction. However, this problem can become challenging if the kinetic parameters are time-varying due to continuous changes of catalyst and enzyme reactivity. Using machine learning methods, in Chapters 3 and 9, we have demonstrated that hybrid modelling provides an effective solution to account for the time-varying nature of kinetic parameters, reducing the model uncertainty. However, another longstanding challenge for predictive modelling of complex chemical and biochemical reactions is their history-dependent behaviour. For example, bistable reactions have ‘memory’ and their trajectories are dependent on past process conditions. Simultaneously resolving history-dependent kinetic model structure identification and time-varying parameter estimation has rarely been studied due to the complexity of the underlying mechanisms and lack of efficient mathematical optimisation algorithms. Therefore, in this chapter, we adopt reinforcement learning (RL) to resolve this challenge by integrating it with hybrid modelling. This chapter introduces a novel three-step modelling framework: (i) speculate and combine possible kinetic model structures sourced from process and phenomenological knowledge, (ii) identify the most likely kinetic model structure and its parameter values using RL and (iii) validate the hybrid model identified. To demonstrate the applicability of the framework, in silico experiments explore three different biochemical scenarios. The results show that the proposed framework efficiently constructs hybrid models to quantify both time-varying and history-dependent kinetic behaviours while minimising the risks of over-parametrisation and over-fitting, highlighting the potential of this framework for general chemical and biochemical reaction modelling.