Recognizing semantic correlation in image-text weibo via feature space mapping

Abstract

Recent years have witnessed the fast development of social media platforms, such as Twitter, Sina Weibo, and Wechat. Practically, the textual weibos are frequently uploaded with images, namely image-text weibos in this paper. To gain the deep insights into the semantics of the image-text weibos, this paper explores the semantic correlation between the image and text. The semantic correlation recognition approach based on feature space mapping and support vector machine has been developed, due to the heterogeneity and incomparability of image, text, and social multi-source information in image-text weibos. Our model firstly extracts three types of features, namely, textual-linguistic, visual, and social features. It then uses the genetic algorithm to project the features from the different feature spaces to the unified one. At last, the semantic correlation recognition model based on support vector machine is constructed in the unified feature space. The experimental results show that the accuracy of our recognition model for semantic correlation between image and text in image-text weibo, with feature space mapping and support vector machine using the three types of multi-source features, achieves a significant performance compared to the traditional model only based on support vector machine.