Polysemious visual representation based on feature aggregation for large scale image applications

Abstract

Multiple image features and multiple semantic concepts from the images have intrinsic and complex relations. These relations influence the effectiveness of image semantic analysis methods, especially on the large scale problems. In this paper, a framework of generating polysemious image representation through three levels of feature aggregation is proposed. In the codebook level aggregation, visual dictionaries are learned for each feature type, and each image feature can be reconstructed with this dictionary. In the semantic level aggregation, the multiple concept distributions are learned with each feature codebook by using the improved local anchor embedding. Then the polysemious representation for for single feature type can be established after this level. In the multiple feature level aggregation, final image polysemious representation is obtained through multiple feature fusion with a weighted pooling approach. Through the proposed framework, multiple feature fusion and multiple semantic descriptions are both achieved in an integrated way. Experimental evaluations on large scale image dataset validate the effectiveness of the proposed method.