Likelihood-based generalization of Markov parameter estimation and multiple shooting objectives in system identification

Abstract

This paper considers system identification (ID) of linear and nonlinear non-autonomous systems from noisy and sparse data. We analyze an optimization objective derived from Bayesian inference for the dynamics of hidden Markov models. We then relate this objective to that used in several state-of-the-art approaches for both linear and nonlinear system ID. In the former, we analyze least squares approaches for Markov parameter estimation, and in the latter, we analyze the multiple shooting approach. We demonstrate the limitations of the optimization problems posed by these existing methods by showing that they can be seen as special cases of the proposed optimization objective under certain simplifying assumptions: conditional independence of data and zero model error. Furthermore, we observe that the proposed approach has improved smoothness and inherent regularization that make it well-suited for system ID and provide mathematical explanations for these characteristics’ origins. Finally, numerical simulations demonstrate a mean squared error over 8.7 times lower compared to multiple shooting when data are noisy and/or sparse. Moreover, the proposed approach identifies accurate and generalizable models even when there are more parameters than data or when the system exhibits chaotic behavior.