Trigraph Regularized Collective Matrix Tri-Factorization Framework on Multiview Features for Multilabel Image Annotation

Abstract

Due to the explosive growth of image data, image annotation has been one of the most popular research directions in computer vision. It has been widely used in image retrieval, image analysis and understanding. Because traditional manual image annotation is time consuming, more advanced automatic annotation methods are needed. A major challenge in developing an efficient image annotation method is how to effectively use all available information contained in the data. To this end, this paper proposes a novel image annotation framework that uses multiple information from data. It employs nonnegative matrix tri-factorization (NMTF) to simultaneously factorize image-to-label, image-to-feature, and feature-to-label relation matrices using their intertype relationships and incorporates the intratype information through manifold regularizations. This method can be referred to as the trigraph regularized collective matrix tri-factorization framework (TG-CMTF). TG-CMTF captures the correlations among different labels, different images and different features. By taking advantage of these relations from images, features and labels, TG-CMTF can achieve better annotation performance than most state-of-the-art methods. The promising experimental results on three standard benchmarks have shown the effectiveness of this information. Furthermore, we show the annotation process as a precise optimization problem and solve it by an iterative algorithm, which proves the correctness of the proposed method from the mathematical theory.