Method to design a neural network with minimal number of neurons for approximation problems

Abstract

The widespread use of neural networks to model complex processes requires that a parsimonious model of the process be obtained. One of the main variables in neural networks is the number of neurons in the hidden layer. Selecting an inappropriate number of neurons can lead to over- or underfitting. Therefore, a method is required which determines the appropriate number of neurons in order to approximate a defined system response or time function. This paper presents a proposition to determine the appropriate number of neurons in a feedforward neural network, based on the number of inflection points included in the system response or the time function. The results show that the proposed method has marginal approximation errors (no underfitting) and overfitting can never occur because the minimal number of neurons for the approximation problem is used. To verify the effectiveness of this method, simulations were carried out on a second-order system with and without noise, the Lotka-Volterra equations, and the Runge function.