Training Accurate Binary Neural Networks from Scratch

Abstract

Binary neural networks are a promising approach to execute convolutional neural networks on devices with low computational power. Previous work on this subject often quantizes pretrained full-precision models and uses complex training strategies. In our work, we focus on increasing the performance of binary neural networks by training from scratch with a simple training strategy. In our experiments we show that we are able to achieve state-of-the-art results on standard benchmark datasets. Further, we analyze how full-precision network structures can be adapted for efficient binary networks and adopt a network architecture based on a DenseNet for binary networks, which lets us improve the state-of-the-art even further. Our source code can be found online: https://github.com/hpi-xnor/BMXNet-v2