Effect of injected noise in deep neural networks

Abstract

Deep Neural Networks have become increasingly popular due to their efficient realization in GPU hardware. Problems that were once considered computationally intensive to implement using Neural networks have now become possible due to the vast amount of flexibility and capability offered by the GPU and Deep networks combination. In this work, we attempt to improve the recognition rate for images, using Deep Neural Networks for the classification task. The Artificial Neural Network is modeled on the biological Neural network, where many thousands of neurons and dendrites act synchronously to perform the recognition task. One aspect of this network is the inherent noise in the signals, due to the physical proximity of electrical connections, and hence through inductive and capacitive coupling. In this work, we have used this as a motivation to add a noise component to the values through the network, thereby to create a more realistic model of the biological neural network. We have constructed a neural network architecture in which the output of neurons in each layer is affected by the value of the signal in the neighboring neurons. This method has the predicted effect of reducing overfitting and further increases the Recognition Rate for Deep Networks by up to 8 percent.