Learning image convolutional representations and complete tags jointly

Abstract

In this paper, we develop a novel image tag completion method. We propose to represent the images by using the convolutional neural network (CNN) and predict the complete tags from the convolutional representations. The prediction is performed by a linear predictive model, and the complete tags are also imposed to be consistent to the existing elements of the incomplete tag matrix. We propose to learn the CNN parameters, the complete tags, and the predictive model parameters jointly. The learning problem is modeled by a minimization problem of an objective function composed of a consistency term between the learned complete tag vectors and the existing incomplete tag matrix, a prediction error term, and the convolutional similarity regularization term, and a sparsity term of the complete tag vector. The minimization problem is solved by an augmented Lagrangian method. The experiments over some benchmark data sets show that our method outperforms the state-of-the-art image tag completion methods.