Abstract
Recent years have brought an unprecedented and rapid development in the field of Natural Language Processing. To a large degree this is due to the emergence of modern language models like GPT-3 (Generative Pre-trained Transformer 3), XLNet, and BERT (Bidirectional Encoder Representations from Transformers), which are pre-trained on a large amount of unlabeled data. These powerful models can be further used in the tasks that have traditionally been suffering from a lack of material that could be used for training. Metaphor identification task, which is aimed at automatic recognition of figurative language, is one of such tasks. The metaphorical use of words can be detected by comparing their contextual and basic meanings. In this work, we deliver the evidence that fully automatically collected dictionary definitions can be used as the optimal medium for retrieving the non-figurative word senses, which consequently may help improve the performance of the algorithms used in metaphor detection task. As the source of the lexical information, we use the openly available Wiktionary. Our method can be applied without changes to any other dataset designed for token-level metaphor detection given it is binary labeled. In the set of experiments, our proposed method (MIss RoBERTa WiLDe) outperforms or performs similarly well as the competing models on several datasets commonly chosen in the research on metaphor processing.
Highlights
MelBERT’s design allows for using the principles of MIP (Metaphor Identification Procedure) [12] simultaneously with the concept of SPV (Selectional Preference Violation) [26], both of which we describe in detail in the following subsection
While MelBERT is not the first model following the guidelines of MIP or SPV in metaphor detection, we found the fact of complementing linguistic theory with the power of recently published bidirectional language models appealing and decided to further build on MelBERT’s authors’ ideas
We present the architecture of MIss RoBERTa WiLDe, a model for Metaphor Identification using the RoBERTa language model
Summary
We present the architecture of MIss RoBERTa WiLDe, a model for Metaphor Identification using the RoBERTa language model. MIss WiLDe utilizes MelBERT (Metaphor-aware late interaction over BERT) published recently by Choi et al [10] and the architecture of the two models is almost identical. For the model overview, whose design was inspired by the aforementioned work, see Figure 1. MIss WiLDe and MelBERT take advantage of the same linguistic methods for metaphor detection, namely SPV (Selectional Preference Violation) and MIP (Metaphor Identification Procedure). While the implementation of the former in our model remains mostly unchanged, the latter is affected by a different kind of input, which is the first novelty of our approach
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
