Fusing cross-media for topic detection by dense keyword groups

Abstract

Events are real-world occurrences that lead to the explosive growth of web multimedia content such as images, videos and texts. Efficient organization and navigation of multimedia data in the topic level can boost users׳ understanding and enhance their experience of the events that have happened. Due to the potential application prospects, multimedia topic detection has been an active area of research with notable progress in the last decade. Traditional methods mainly focus on single media, so the results only reflect the characteristics of one certain media and topic browsing was not comprehensive enough. In this paper, we propose a method of utilizing and fusing rich media information from web videos and news reports to extract weighted keyword groups, which are used for cross-media topic detection. Firstly by utilizing the video-related textual information and the titles of news articles, a maximum local average score is proposed to find coarse weighted dense keyword groups; after that, textual linking and visual linking are applied to refine the keyword groups and update the weights; finally, the documents are re-linked with the refined keyword groups to form an event-related document set. Experiments are conducted on cross-media datasets containing web videos and news reports. The web videos are from Youku, YouTube׳s equivalent in China, the news reports from sina.com, some of which contain topic-related images. The experimental results demonstrate the effectiveness and efficiency of the proposed approach.