Neural network initialization with prototypes - a case study in function approximation

Abstract

The initialization of neural networks in function approximation has been studied by many researchers yet remains a challenging problem. Another important yet open issue in the neural network community is to incorporate knowledge and hints with regard to training for a meaningful neural network. This study makes an attempt to address these two issues in handling a specific type of engineering problems, namely, modeling nonlinear hysteretic restoring forces of a dynamic system under a specific formulation. The paper showcases a heuristic idea on using a growing technique through a prototype-based initialization where the insights to the governing mathematics/physics are related to the features of the activation functions.