Multi-Modal Topic Model Based on Word Rank and Relevance Semantic for Social Events Classification

Abstract

Social event classification of multimedia is a hot issue in the field of multimedia research. The existing social event classification methods based on supervised topic model fail to make full use of the internal semantic information (text, vision, etc.) in the corpus, and the classification performance of the model can be further improved. To solve this problem, a multi-modal supervised topic model (multi-modal supervised topic model based on word rank and relevance semantic weighting, DPRF-MMSTM) is proposed, which integrates word rank semantics and word document relevance semantics. According to the results of dependency parsing, the contribution of text modal words to document representation can be divided, and the hierarchical semantics of text words can be mined. In addition, the relevance frequency of multi-modal words is considered to extract the relevance semantics of word documents. The two semantics are integrated into the sampling process of multi-modal words to achieve the classification of social events based on supervised topic model. Compared with the existing models, the comparative experiments on multi-modal and text modal datasets show that the DPRF-MMSTM model proposed in this paper improves the classification accuracy of social events by 1.200% and 1.630% respectively, and the topic consistency is increased by 38.0% and 8.5% respectively.