An evidence-based multimodal fusion approach for predicting review helpfulness with human-AI complementarity

Abstract

Online product reviews have been increasingly adopted by consumers to express their evaluation for products on e-commercial platforms. Consumers are also encouraged to evaluate the quality of reviews by a voting mechanism, according to their human intelligence. Considerable machine learning approaches have been developed to help detect helpful reviews automatically. However, previous studies largely depend on machine intelligence and neglect the complementary strength between machine algorithms and human intelligence. This study proposes an evidence-based multimodal fusion approach for predicting the helpfulness of reviews by aggregating the human intelligence from user votes and machine intelligence from texts and images. Review helpfulness is measured by a belief distribution and represented by the evidence obtained from each modality data. An evidential reasoning rule-based fusion strategy is proposed to combine the hybrid intelligence considering their complementarity. A new dataset is also developed for the task of multimodal review helpfulness prediction by crawling online product reviews from Amazon.com and annotated by human scoring. Experimental results on the dataset show that the proposed approach for predicting review helpfulness has superior predictive power and outperforms existing methods. This study also contributes to e-commerce by providing an enhanced method for predicting review helpfulness from both human and machine intelligence.