Nonlinear system identification using a recurrent network in a Bayesian framework

Abstract

Modern deep neural networks are being widely exploited to solve challenging learning tasks, including nonlinear system identification. Bayesian system identification intrinsically encapsulate uncertainty in model parameters and provides forecasting distribution enabling enhanced analysis, simulation and control system design. Nevertheless, the application of the full Bayesian approach to articulated models as deep neural networks results quite challenging in practice. In this work we propose an identification technique for nonlinear dynamic systems exploiting a deep recurrent neural network with Long-Short Term Memory (LSTM) units retaining a Bayesian framework. To such an aim, we stacked the recurrent neural network with a probabilistic layer, decomposing the nonlinear dynamic model into a combination of flexible functions. Hence, deterministic and stochastic layers are trained jointly, forcing the learning algorithm to transform the input data sequences into a deterministic feature space encoded by the LSTM, useful for predictions. Besides, we deployed a scalable technique based on Variational Inference to deal with the exact inference intractability. We show the effectiveness of the proposed approach by the application to a widely exploited open benchmark for nonlinear system identification.