Conceptual commonsense-aware attentive modeling with pre-trained masked language models for humor recognition

Abstract

Humor is an important component of daily communication and usually causes laughter that promotes mental and physical health. Understanding humor is sometimes difficult for humans and may be more difficult for AIs since it usually requires deep commonsense. In this paper, we focus on automatic humor recognition by extrapolating conceptual commonsense-aware modules to Pre-trained Masked Language Models (PMLMs) to provide external knowledge. Specifically, keywords are extracted from an input text and conceptual commonsense embeddings associated with the keywords are obtained by using a COMET decoder. By using multi-head attention the representations of the input text and the commonsense are integrated. In this way we attempt to enable the proposed model to access commonsense knowledge and thus recognize humor that is not detectable only by PMLM. Through the experiments on two datasets we explore different sizes of PMLMs and different amounts of commonsense and find some sweet spots of PMLMs’ scales for integrating commonsense to perform humor recognition well. Our proposed models improve the F1 score by up to 1.7% and 4.1% on the haHackathon and humicroedit datasets respectively. The detailed analyses show our models also improve the sensitivity to humor while retaining the predictive tendency of the corresponding PMLMs.