Abstract
Extracting useful knowledge from multimodal data is the core of many multimedia applications, such as recommendation systems, and cross-modal retrieval. In this paper, we propose a label-based multimodal topic (LB-MMT) model to jointly model text and image data tagged with multiple labels. Specifically, we use the labels as supervised information to generate the text and image data. In the LB-MMT model, we assume that the textual words and visual words related to each text and image are drawn from a mixture of latent topics, where each topic is represented as a group of textual words and visual words. Moreover, we introduce multiple topics for each label, to build the top-down relationship from label to text and image. To investigate the effectiveness of the proposed approach, we conduct extensive experiments on a real-world multimodal dataset with labels. The results show the proposed approach obtains superior performances on topic coherence and label prediction compared with previous competitors. In addition, we show that our model yields interesting insights about multimodal topics. The proposed model provides important practical implications, e.g., designing more attractive multimodal contents for marketers.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.