Image semantic learning method based on social heterogeneous graph networks1

Abstract

Image semantic learning techniques are crucial for image understanding and classification. In social networks, image data is widely disseminated thanks to convenient acquisition and intuitive expression characteristics. However, due to the freedom of users to publish information, the image has apparent context dependence and semantic fuzziness, which brings difficulties to image representation learning. Fortunately, social attributes such as hashtags carry rich semantic relations, which can be conducive to understanding the meaning of images. Therefore, this paper proposes a new method named Social Heterogeneous Graph Networks (SHGN) for image semantic learning in social networks. First, a heterogeneous graph is built to expand image semantic relations by social attributes. Then the consistent semantic space is reconstructed through cross-media feature alignment. Finally, an image semantic extended learning network is designed to capture and integrate the social semantics and visual feature, which obtains a rich semantic representation of images from a social context. The experiments demonstrate that SHGN can achieve efficient image representation, and favorably against many baseline algorithms.