Difference networks and second-order difference networks

Abstract

Deep convolution networks [1] showed extraordinary talents in computer vision in recent studies. However, how to train or optimize it is still difficult. In this paper, we introduce the Difference Network (DiffNet) and second-order Difference Network (soDiffNet), which propagated “salient difference features” with various identity shortcut connections. We embrace difference equations which combined current stack layers and input layers, rather than unreferenced functions. Furthermore, to utilize the high potential of identity shortcut connection, we introduced the second-order difference network unit to add depth of unit and therefore to the whole network. Notably, the difference network and the second-order network can be applied to diverse kinds of convolutional networks and significantly increase the performance over a variety of applications. In summary, our experiments on benchmark CIFAR-10, CIFAR-100, and CASIA-WebFace datasets investigated have shown that our network architecture has effectiveness and versatility in object classification and face recognition.