New method of training two-layer sigmoid neural networks using regularization

Abstract

We propose a complex learning algorithm for sigmoid Artificial Neural Networks (ANN). We introduce the concept of the working area of a neuron for sigmoid ANNs in the form of a band in the attribute space, its width and location associated with the center line of the band to a fixed point. We define of the centers and widths of the working areas of neurons by analogy to the radial ANNs. On this basis, an algorithm for selecting the initial approximation of network parameters, ensuring uniform coverage of the data area with neuron working areas was developed. Network learning is carried out using a non-smooth regularizer designed to smooth and remove non-informative neurons. The results of the computational experiment illustrate the efficiency of the proposed integrated approach.