Nonlinear matrix factorization with unified embedding for social tag relevance learning

Abstract

With the proliferation of social images, social image tagging is an essential issue for text-based social image retrieval. However, the original tags annotated by web users are always noisy, irrelevant and incomplete to interpret the image visual contents. In this paper, we propose a nonlinear matrix factorization method with the priors of inter- and intra-correlations among images and tags to effectively predict the tag relevance to the visual contents. In the proposed method, we attempt to discover the image latent feature space and the tag latent feature space in a unified space, that is, each image or each tag can be described as a point in the unified space. Intuitively, it is more understandable to estimate the relationships between images and tags directly based on their distances or similarities in the unified space. Thus, the task of image tagging or tag recommendation can be efficiently solved by the nearest tag-neighbors search in the unified space. Similarly, we can obtain the top relevant images corresponding to any tag so as to perform the task of image search by keywords. We investigate the performance of the proposed method on tag recommendation and image search respectively and compare to existing work on the challenging NUS-WIDE dataset. Extensive experiments demonstrate the effectiveness and potentials of the proposed method in real-world applications.