Cross-media retrieval based on semi-supervised regularization and correlation learning

Abstract

As large scale multimedia data in heterogeneous spaces is flooding into the Internet, cross-media retrieval is becoming increasingly significant. In cross-media retrieval, users can retrieve the results containing various types of media by submitting a query of any media type. However, most existing cross-media retrieval methods are restricted to the retrieval between two types of media, which ignores the semantic consistency of different media data. In addition, although some methods consider the similarity between same semantic category data in different media, they neglect the dissimilarity between different semantic category data in different media. To solve the above problems, we propose a novel feature learning algorithm for cross-media retrieval, called semi-supervised regularization and correlation learning (SSRCL), which is capable of modeling multiple types of media simultaneously. More importantly, SSRCL considers both semantic category similarity and dissimilarity simultaneously, and utilizes both labeled and unlabeled data to learn the projection matrices for different media types. The experimental results show that our proposed approach, compared with four state-of-the-art methods, has better performance on two extensively used datasets.