A co-attention based multi-modal fusion network for review helpfulness prediction

Abstract

The current review helpfulness prediction (RHP) methods simply rely on the textual features and meta features to predict review helpfulness, overlooking the informational value of images. Besides, hand-crafted and deep features of text and images have unique advantages, but the combination of them is rarely considered in previous studies. To address these issues, this paper proposes a novel end-to-end architecture utilizing hand-crafted and deep features of text and images simultaneously for RHP. First, the self-attention mechanism considers the intra-modal correlation between hand-crafted and deep features by weighting features at all positions of text and images. Second, a co-attention mechanism is designed to explore dependencies between text and image modality. Third, multi-modalities are fused by simultaneously considering intra-modal and inter-modal interactions for helpfulness prediction. Our proposed framework is verified by two real-world datasets collected from Yelp.com and Amazon.com respectively. The experimental results confirm the favorable performance of our model compared with the benchmark methods. The findings of this study are expected to raise attention to images laden in online reviews, and the complementarity between texts and images from scholars and practitioners.