Explainable AI for Multimodal Credibility Analysis: Case Study of Online Beauty Health (Mis)-Information

Abstract

“One person’s data or experience is another person’s information” this has become the golden rule of the 21st century which has resulted in a massive reservoir of data and immense amounts of information generation. However, there is no control over the source of this information, accessibility of this information, or the quality of it, which has given rise to the presence of “misinformation.” The research community has reacted by proposing frameworks and difficulties, which are helpful for (different subtasks of) recognizing misinformation. Most of these frameworks, however, fail to consider all the aspects that can contribute to making information “credible”. Furthermore, a valid explanation for each considered feature’s contribution to the model’s decision stands missing in most work. With this in mind, the authors have attempted to produce a system that yields highly accurate decisions, thus effectively separating credible health blogs from their non-credible counterparts while providing valid user-friendly explanations. The study proposes an Explainable AI-assisted Multimodal Credibility Assessment System that examines the credibility of the platform where the blog is hosted, the credibility of the author of the blog and the credibility of the images that contribute to the blog. This novel framework contributes to the existing body of knowledge by assessing the credibility of misleading beauty blogs using multiple crucial modalities which would lead to an insightful information consumption by the users. The proposed pipeline was successfully implemented on multiple carefully curated datasets and correctly identified 274 non credible blogs out of 321 blogs with an accuracy of 97.5%, Precision of 0.973 & F1score of 0.986. Further, the Explainable AI model, with the help of several visualizations displayed the feature contributions for each blog & it’s impact and magnitude in a concise comprehensible format. The framework can be further customized and applied to various domains where presence of misinformation is of high concern such as pharmaceutical drug information, pandemic management, financial advisories, online healthcare services and cyber frauds.