NLPRL-IITBHU at SemEval-2018 Task 3: Combining Linguistic Features and Emoji pre-trained CNN for Irony Detection in Tweets

Harsh Rangwani,Devang Kulshreshtha,Anil Kumar Singh

doi:10.18653/v1/s18-1104

Abstract

This paper describes our participation in SemEval 2018 Task 3 on Irony Detection in Tweets. We combine linguistic features with pre-trained activations of a neural network. The CNN is trained on the emoji prediction task. We combine the two feature sets and feed them into an XGBoost Classifier for classification. Subtask-A involves classification of tweets into ironic and non-ironic instances whereas Subtask-B involves classification of the tweet into - non-ironic, verbal irony, situational irony or other verbal irony. It is observed that combining features from these two different feature spaces improves our system results. We leverage the SMOTE algorithm to handle the problem of class imbalance in Subtask-B. Our final model achieves an F1-score of 0.65 and 0.47 on Subtask-A and Subtask-B respectively. Our system ranks 4th on both tasks respectively, outperforming the baseline by 6% on Subtask-A and 14% on Subtask-B.

Highlights

According to the Merriam-Webster dictionary1, one of the meanings of irony is defined as ‘the use of words to express something other than and especially the opposite of the literal meaning’ (e.g. I love getting spam emails.)
Leveraging DeepMoji model for Irony detection domain yields a considerable improvement over purely linguistic features (0.03 and 0.12)
We reported the use of handcrafted features and pre-trained Convolutional Neural Network (CNN) activations for predicting the irony in tweets

Summary

Introduction

According to the Merriam-Webster dictionary, one of the meanings of irony is defined as ‘the use of words to express something other than and especially the opposite of the literal meaning’ (e.g. I love getting spam emails.). Irony can have different forms, such as verbal, situational, dramatic etc. Sarcasm is categorized as a form of verbal irony. Sarcastic texts are characterized by the presence of humor and ridicule, which are not always present in the case of ironic texts (Kreuz and Glucksberg, 1989). The absence of these characteristics makes automatic irony detection a more difficult problem than sarcasm detection

Methods

Results

Conclusion