Singularities in the identification of dynamic systems

Abstract

As is well known, the parameter spaces of hierarchical systems such as multilayer perceptrons include singularities and the plateau phenomenon is ubiquitous in the process of learning. In the singular regions, the Fisher information matrix degenerates and the loss function is almost unchanged when the parameters arrive in the singular regions, which is called the plateau phenomenon. We wonder about whether the singularities and the plateau phenomenon exist in the parameter identification process of the linear and the ordinary nonlinear systems. In this paper, we can see that in some of the parameter identification of the nonlinear systems, the Fisher information matrix degenerates, the singularities exist and we can see the plateau phenomenon in the learning curves. A simulation example is provided to demonstrate the theoretical analysis in Section 3.