Adaptive stochastic resonance based convolutional neural network for image classification

Abstract

In this paper, we exploit the adaptive stochastic resonance effect in the convolutional neural network with threshold activation functions for enabling the back-propagation gradient computation. During training, the injection of noise into the threshold activation function makes it to be differentiable at forward pass, thus exact gradients of the loss function with respect to network parameters including the injected noise level can smoothly propagate at backforward pass. This training method effectively uses the non-zero noise to improve the performance of the designed convolutional neural network. In the testing phase, the differentiable activation function is decoupled into a set of threshold functions driven by mutually independent noise components, which endows a hardware-friendly feature of the designed neural network. Experiments show that the designed low-precision convolutional neural network has a comparable performance with the full-precision one with ReLU activation functions on two benchmark data sets. These results also extend the potential application of adaptive stochastic resonance into the convolutional neural network for image classification.