Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling

Abstract

Listen

Translate article

Stochastic sampling is performed to reduce hardware energy consumption and prevent overfitting by reducing parameters, because not all data are required for learning. In this study, a new approach, pseudo‐synaptic sampling (PS2) method, which approximates the conventional synaptic sampling machine (S2M) method through a hardware‐friendly implementation while demonstrating superior efficiency, is introduced. By sampling in front of neurons rather than at each synapse, the PS2 method improves hardware energy efficiency and ensures scalability. Furthermore, it improves energy and area efficiency by eliminating the additional circuit required by other techniques, such as the random walk (RW) method previously used which requires an additional circuit to frequently charge/discharge the membrane potential. Herein, the average firing rate equation for the S2M method is modified to suit the experimental conditions of this study. Through this numerical simulations, it is confirmed that the activation function of the PS2 method aligns with that of the S2M method and verified that the PS2 method can implement stochasticity for restricted Boltzmann machine (RBM) neurons. Experimental validation of the PS2 method, compared to the RW method, for Modified National Institute of Standards and Technology database (MNIST) training and inference on field‐programmable‐gate‐array‐implemented spiking RBM chips reveals promising results. In an MNIST 100‐handwritten digit experiment, the PS2 method exhibits on‐chip training accuracy (92%) comparable to that of the RW method (93%). Furthermore, in the energy consumption analysis, it is shown that the PS2 method reduces power consumption by 94.94% compared to the RW method, highlighting its enhanced power efficiency due to a reduced number of circuit elements. In the investigation into the impact of increasing the frequency at which random bits are generated, it is shown that the RW method experiences accuracy degradation even with slight increases, whereas the PS2 method maintains accuracy over significantly longer periods. This enables further power reduction by allowing for a longer period during random bit generation. In this study, a foundation is laid for maximizing the energy efficiency of spiking neural network processors by optimizing internal noise generation mechanisms.