A novel localized and second order feature coding network for image recognition

Abstract

Vector of Locally Aggregated Descriptor (VLAD) is a very popular feature coding method in image classification and image retrieval. Recently, the original VLAD method is extended to an end-to-end model called NetVLAD. The NetVLAD layer is readily embedded into a deep neural network and can be trained by the back-propagation algorithm. Although the NetVLAD model has achieved noticeable classification results in many image databases, the discrimination embedded in the NetVLAD method is not fully exploited. In this paper, in order to design a more discriminative feature coding network, a novel localized and second-order VLAD Network (LSO-VLADNet) is proposed. First, we design a localized and second-order VLAD coding method. Second, the back propagation functions of all newly designed layers are obtained. Third, the new feature coding method is extended to an end-to-end feature coding network which can be jointly trained with a deep convolutional neural network for visual recognition. Some experiments show that the newly designed network has the significant improvements over the original NetVLAD. Some experimental comparisons of the proposed model and other state-of-art methods will also be given to validate the effectiveness of the proposed model.