StoryRoleNet: Social Network Construction of Role Relationship in Video

Abstract

The automatic extraction of social relationship among individuals from massive quantities of video data is an important aspect of information extraction. However, most existing studies have focused on rough information, which result in inaccurate social network of role relationship. In this paper, the StoryRoleNet model is proposed for constructing an accurate and integral network representing the relationships among roles. First, to avoid the redundancy calculation of the relationships on the segmentation points of neighboring story units, we measure the weights of relationships by a weighted-Gaussian method in each story unit. More importantly, a new story segmentation method for long video is proposed by analyzing hierarchical features of the video. Then, we combine relationship networks constructed from the video and subtitle text. Some missed relationships can be complemented by this way. At last, the final network is analyzed to discover communities and important roles. Comprehensive evaluations were conducted using three movies and one television drama. The results demonstrate that the proposed method outperforms state-of-the-art methods in terms of the $F_{1}$ accuracy measure and the normalized mutual information value.