Learning part-based mid-level representation for visual recognition

Abstract

Abstract There exists a huge semantic gap between the low-level image representations and high-level semantics. To bridge such a gap, this paper proposes a mid-level image representation for visual recognition, where an image is represented based upon the response maps of local part filters. Each dimension of the mid-level representation indicates the likelihood of seeing a part in the input image. The part filters are trained using external data and need not to be fine-tuned on test data. To eliminate the possibly redundant similar parts occurring in different objects or scenes, we perform unsupervised clustering for part refinement. To alleviate the expensive computation of the response maps of the part filters, we further leverage sparse coding to accelerate the feature extraction process, which is ten times faster without significantly compromising the recognition accuracy. We evaluate the proposed mid-level representation on both image and video content recognition tasks and attain state-of-the-art results.