Effect of stochastic activation function on reconstruction performance of restricted Boltzmann machines with stochastic magnetic tunnel junctions

Abstract

Stochastic Magnetic Tunnel Junctions (SMTJs) emerge as a promising candidate for neuromorphic computing. The inherent stochasticity of SMTJs makes them ideal for implementing stochastic synapses or neurons in neuromorphic computing. However, the stochasticity of SMTJs may impair the performance of neuromorphic systems. In this study, we conduct a systematic examination of the influence of three stochastic effects (shift, change of slope, and broadening) on the sigmoid activation function. We further explore the implications of these effects on the reconstruction performance of Restricted Boltzmann Machines (RBMs). We find that the trainability of RBMs is robust against the three stochastic effects. However, reconstruction error is strongly related to the three stochastic effects in SMTJs-based RBMs. Significant reconstruction error is found when the stochastic effect is strong. Last, we identify the correlation of the reconstruction error with each stochastic factor. Our results might help develop more robust neuromorphic systems based on SMTJs.