Projective nonnegative matrix factorization for social image retrieval

Abstract

Increasingly many social images with tags are available on photo sharing websites. Due to the subjectivity and diversity of social tagging behaviors, noisy and missing tags for images are inevitable. To tackle this problem, this paper proposes a novel factor analysis model, named ProjecTive Nonnegative Matrix Factorization (PTNMF) with ℓ2,1-norm regularization, which introduces linear transformation and ℓ2,1-norm minimization into a joint framework of NMF. For tagging data, a new interpretation is adopted to distinguish the relevant tags and irrelevant tags instead of the typically used binary scheme. In our model, the image latent representation is assumed to be projected from its original feature representation with an orthogonal transformation matrix. The projection makes convenient to embed any images including out-of-samples into the latent space. That is, the proposed method enables to handle the out-of-sample problem. The ℓ2,1-norm regularization makes the transformation matrix suitable for selecting the effective features. Local geometry preservations of image space (tag space) are explored as constraints in order to make image similarity (tag correlation) consistent in the original space and the corresponding latent space. We investigate the performance of the proposed method on image retrieval and compare it to existing work on the challenging NUS-WIDE dataset. Extensive experiments indicate the effectiveness and potentials of the proposed method in real-world applications.