Modeling Nonlinear Dynamic Systems Using BSS-ANOVA Gaussian Process

Abstract

Nonlinear dynamic systems are some of the most common variety of systems encountered in the sciences, but are the potentially more onerous to model through system identification than static systems due to their added complexity, sensitivity to initial conditions, and the potential application of new dynamic and nonlinear behavior through any time dependent forcing functions. The BSS-ANOVA Gaussian Process is a Machine Learning method for dynamic system ID that possesses several attributes that make it a natural candidate for this variety of problem. BSS-ANOVA is fully Bayesian, works best for continuous tabular datasets, and fast training and inference times and high model fidelity. The BSS-ANOVA GP is based upon a Karhunen- Loeve (KL) expansion of the basis set of the BSS-ANOVA kernel. This produces ordered, non-parametric, and spectral terms that are capable of utilizing Gibbs sampling during model building. Each of the model’s terms can be viewed as either a main effect of an input or as the interaction between two to three inputs coupled with a weight. These terms can be added iteratively to the model using forward variable selection, beginning with the lowest order effects, until the model’s accuracy is balanced by its complexity via a cost function. For dynamic systems, the GP creates a model of the static derivatives of the system, then integrates. This approach is carried out for three sets of data. The first creates a model of a benchmark dataset concerning the heights of a pair of cascaded tanks fed by a pump provided a random current, viewed here as a forcing function. The second models a synthetic dataset of SIR disease transmission, which has a constant transmissibility input during training and a variable transmissibility for inference. The third models the degradation rate of a solid oxide fuel cell as a function of its operating temperature, current density, and overpotential. The former two datasets are compared to a variety of alternative ML methods. The static derivative problem is compared to the results of a Random Forest, a Residual Neural Network, and an Orthogonalized Additive Kernel (OAK) inducing point GP. The dynamic time series predictions are compared to the GRU and LSTM Recurrent Neural Networks (RNN), as well as SINDy. For the static predictions, the BSS-ANOVA model exceeds the accuracy of the Residual Neural Network and the Random Forest, with lower accuracy than the the OAK GP while being several orders of magnitude faster. SINDy