Stochastic weights binary neural networks on FPGA

Abstract

To achieve an advanced Internet of Things (IoT), it is necessary to combine artificial intelligence (AI) with IoT. Compact circuits that can operate AI functions will be useful for this purpose. Therefore, we propose stochastic weights binary neural networks (SWBNN). SWBNNs are more accurate than binary neural networks (BNN) with small circuits. BNNs can be realized with small circuits since binary calculation needs simpler circuits than real number calculation. However, BNNs have lower accuracy than networks with real numbers. Thus, the proposed SWBNNs are BNNs that behave stochastically, which makes them more accurate than BNNs. Moreover, SWBNNs can still be achieved with small circuits since they execute binary calculation. As a result, the accuracy for the test data of SWBNNs is closer to the accuracy for learning data than the accuracy for the test data of BNNs is. Especially when using the CIFAR10 database, the difference in the identification accuracy rate between learning data and test data decreased from 6% for BNNs to 2% for SWBNNs. From results of a field-programmable gate array (FPGA) implementation, circuits of SWBNNs are sufficiently small although they are 10% bigger than those of BNNs. Therefore, SWBNNs are more accurate than BNNs, and the circuit costs ofintroducing stochastic weights are low.