Efficient methods of initializing neuron weights in self-organizing networks implemented in hardware

Abstract

In this paper, we focus on the topic of an efficient initialization of neuron weights, which is one of key problems in artificial neural networks (ANNs). This problem is important in ANNs implemented as Application Specific Integrated Circuits (ASICs), in which the number of the weights is relatively large. When ANNs are implemented in software, the weights can be easily modified. In contrast, in neural networks realized as ASICs in which due to parallel data processing each neuron is realized as a separate circuit, it is necessary to provide programming and addressing lines to each memory cell containing a weight. This causes a substantial increase in the complexity of such systems. In this study, we performed comprehensive investigations, in which we simulated the training process of the Self-Organizing ANN with different initialization scenarios. The aim of these investigations was to find simple and efficient initialization procedures that lead to optimal learning process for a broad spectrum of values of other network parameters.The investigations have shown that Self-Organizing Maps (SOMs) in many situations may be trained without any initialization (with zeroed weights). This is possible due to the neighborhood mechanism that to some degree stimulates the neurons belonging to the SOM. We present selected results of several thousands simulations for different topologies of the SOM, for different neighborhood functions and two distance measures between the learning patterns and neurons in the input data space. Simulations were performed for initial values of the weights equal to zero, for small values (up to 1% of full scale range) and for neurons randomly distributed over the overall input data space. The results in most cases are comparable that allows to reduce the complexity of the SOM implemented in the CMOS technology.