Multiple Stage Residual Model for Image Classification and Vector Compression

Abstract

Feature coding is a fundamental issue with many vision tasks, such as image classification, image retrieval and image segmentation, etc. There is no doubt that the encoding procedure leads to information loss, due to the existence of quantization error. The residual vector, defined as the difference between the feature and its corresponding visual word, is the chief culprit to be responsible for the quantization error. Many previous algorithms consider it as a coding issue, and focus on reducing the quantization error by reconstructing the feature with more than one visual word, or by the so-called soft-assignment strategy. In this paper, we consider the problem from a different point of view, and propose an effective and efficient model called multiple stage residual model (MSRM). It makes full use of the residual vector to generate a multiple stage code. MSRM is a hierarchical structure, with the bottom stage producing the coarsest quantization, and the top stage producing the finest quantization. Moreover, our proposed model is a generic framework, which can be built upon many coding algorithms. The interplay of such a coarse-to-fine quantization procedure with a discriminative classifier (e.g., SVM) can improve the classification accuracy of the baseline algorithms significantly. As a special case of MSRM, multiple stage vector quantization (MSVQ) can be directly used for vector compression and approximate nearest neighbor search, and achieves competitive performances with high efficiency.