Image classification algorithm based on social network metadata

Abstract

The images in online social networks usually carry a large amount of social network metadata, such as labels, users, picture groups, locations and comments. These social network metadata, which include rich image semantic information, can help users distinguish the content of imagesand communicate with each other. An image classification algorithm based on multiple social networks, MSNet, is proposed in this paper. Firstly, MSNet collects the social network metadata of images and constructs relationship networks of images based on the metadata. Then, a network embedding algorithm is used to learn the representation vectors of images in each network. Finally, a neural network classifier is trained to classify the images by using the visual features and network representation of images. The experimental results on PASCAL、MIR、CLEF and NUS image data sets show the superiority performance of MSNet in comparison with CNN-neighbor and kernel canonical correlation analysis (KCCA).