The effect of weight precision and range on neural network classifier performance

Abstract

This study investigates the precision and range requirements for weights in feed-forward neural network classifiers using backpropagation training with simulated signals where we could control the difficulty of the problem. We found that a precision of five bits and a range equal to four times the mean weight magnitude produced the same performance as continuous weights. However, we noted that the required precision and range depended strongly on the problem difficulty, the network complexity, and the relationship between these two factors. We also found that uniformly distributed discrete weights produced better performance than non-uniformly distributed ones.