Abstract
Harmful content serves as a potent tool for spreading destructive ideologies and creating divisions among people, often exploiting factors like race, religion, and socioeconomic status. Social media platforms are especially conducive to such content, mainly through memes. Hence, the development of tools that can detect and eliminate harmful memes before they reach a wide audience and cause harm is a very important issue. However, this task represents a critical challenge, as memes can take various forms, typically combining images with text. Recent research has focused on building multimodal machine learning models, especially transformers-based models, to uncover harmful content within memes. Although these models have shown effectiveness, they often require substantial computational resources for training and incorporate additional information besides the textual and visual components of the memes, making them complex and challenging to reproduce. This study introduces a novel approach to enhance the performance of established multimodal transformer models without needing extra information. It works by adding a specific module called Compact Parameter Blocks (CPB) into the encoder segments of these models. This module enhances the quality of input data, leading to improvements in the model’s attention mechanism and in its overall regularization. This results in better performance across all relevant metrics. In this paper, we use this approach with two variations of multimodal models — VisualBERT (Visual Bidirectional Encoder Representations from Transformers) and VilBERT (Vision and Language Bidirectional Encoder Representations from Transformers) — both built on canonical transformers to identify harmful content in memes across four different datasets. The experimental results demonstrated that this method outperforms several robust and complex approaches.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: Engineering Applications of Artificial Intelligence
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.