Hashtag our stories: Hashtag recommendation for micro-videos via harnessing multiple modalities

Abstract

Due to the short attention span and instant gratification phenomenon, micro-videos are growing exponentially while gaining more and more concerns. Yet the sheer number of micro-videos leads to severe information overload issues, making it difficult for users to identify their desired micro-videos. The hashtag, mainly utilized in the domain of the microblog or the image, is the indicator or the core idea of the target content and can be applied to various information retrieval scenarios (e.g., search, browse, and categorization). So far, however, little attention has been paid to perform the hashtag recommendation for micro-videos via harnessing multiple modalities.In this article, we devise a neural network-based solution, LOGO (short for “muLti-mOdal-based hashtaG recOmmendation”), to recommend hashtags for micro-videos by utilizing multiple modalities. The proposed LOGO approach first represents each modality as the combination of sequential units in it, weighted by the attention mechanism. In this way, the sequential and attentive features are captured simultaneously. After that, the LOGO integrates the representations of all modalities via a multi-view representation learning framework, which projects the representations into a common space under the restriction of the modality similarity. Ultimately, the LOGO feed the projections of three modalities in the common space and the embeddings of hashtags into a customized neural collaborative filtering framework to perform the hashtag recommendation. Extensive experiments on the scope of both overall performance comparison and micro-level analyses have well-justified the effectiveness and rationality of our proposed approach.