Improving the accuracy of SqueezeNet with negligible extra computational cost

Abstract

The deep neural network has a strong ability to solve problems in different fields such as computer vision and natural language processing. The development of lightweight models makes neural networks more efficient and can be widely applied to various tasks. SqueezeNet is a typical lightweight neural network composed of fire modules and other common modules in neural networks. However, the original paper achieves only 57.50% and 80.30% Top-1 and Top-5 ImageNet accuracy and uses deep compression, an additional model compression method, to reduce the model size. In this paper, we improve the network from the aspects of training method and network microarchitecture to enhance the practicality of SqueezeNet at negligible extra computational cost. These two improvements significantly improve the performance of the SqueezeNet. Experiments performed on the ImageNet datasets have shown that the proposed method has improved the Top-1 and Top-5 accuracy of SqueezeNet to 64.55% and 85.09%, which means 7.05% and 4.79% relative improvement over the original paper, without additional loss. It only increases the number of parameters by 0.48%. Additionally, the inference speed of our models is almost the same as baseline models on different platforms.