Singularities in the parameter identification of nonlinear dynamic systems

Abstract

Singularity and plateau phenomenon are ubiquitous in the learning process of neural networks. As is known, in the singular regions, the Fisher information matrix degenerates and its inverse G <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> does not exist. The Cramér-Rao theorem is no longer valid at the singular regions. What we want to know is whether the singularities exist in the learning process of the other nonlinear systems and how they affect the learning dynamics. In this paper, for a typical nonlinear system, we give an explicit expression of the Fisher information matrix and find that in the parameter identification of this nonlinear systems, the singularities exist and the plateau phenomenon arises in the learning curve. A simulation example is provided to demonstrate the theoretical analysis in the section 3.