Scale-aware network with scale equivariance

Abstract

Convolutional neural network (CNN) has achieved good performance in object classification due to its inherent translation equivariance, but its ability of scale equivariance is poor. A Scale-Aware Network (SA Net) with scale equivariance is proposed, which can estimate scale, that is, the size of image, while classifying. In the training stage, only one scale pattern is learned. In the testing stage, firstly, the testing sample with unseen scale is zoomed-in and zoomed-out into a set of images with different scales, which form an image pyramid. The image zooming-in channels are up-sampled by bilinear interpolation. The image zooming-out channels are down-sampled, and the combination of dyadic discrete wavelet transform (DWT) and bilinear interpolation are used to avoid spectrum aliasing. Then, the image pyramid with different scales is sent to siamese CNNs with weight-sharing for inferencing. A two-dimensional classification score matrix is obtained. Through the position of the maximum of the classification score matrix, the classification and scale estimation can be carried out at the same time. Experiments are carried out on MNIST Large Scale testing set. In scale estimation experiments, the relative value of root mean square error (RMSE) can be obtained by scaling the testing sample images in a geometric series with common ration of ⁴√2 in the range of [1/2，2]. The classification experiments show that when the scale is greater than 1.0, the classification accuracy can surpass 90%. SA Net can estimate the scale while improving the classification accuracy, and mis-estimated samples are always near the ground-truths (GTs), so the correct scale of the unseen scale can always be obtained roughly.