Abstract
Extracting fine-grained information from social media is traditionally a challenging task, since the language used in social media messages is usually informal, with creative genre-specific terminology and expression. How to handle such a challenge so as to automatically understand the opinions that people are communicating has become a hot subject of research. In this paper, we aim to show that leveraging the pre-learned knowledge can help neural network models understand the creative language in Tweets. In order to address this idea, we present a transfer learning model based on BERT. We fine-turned the pre-trained BERT model and applied the customized model to two downstream tasks described in SemEval-2018: Irony Detection task and Emoji Prediction task of Tweets. Our model could achieve an F-score of 38.52 (ranked 1/49) in Emoji Prediction task and 67.52 (ranked 2/43) and 51.35 (ranked 1/31) in Irony Detection subtask A and subtask B. The experimental results validate the effectiveness of our idea.
Highlights
The social media messages have been commonly used to share thoughts and opinions about the surrounding world and have become a new form of communication [1]
Corpus For the Emoji Prediction Task, we used the corpus provided by SemEval-2018
Effectiveness of Training Epochs Unlike the previous emoji prediction task that contains as large as 500 K training samples, the iron detection task only contains roughly 4 K training samples
Summary
The social media messages have been commonly used to share thoughts and opinions about the surrounding world and have become a new form of communication [1]. Understanding social media messages is not straightforward. With the resurgence of Deep Learning, the recent study of social media understanding mainly focuses on using neural network models. One major problem is that the training process of these models is purely data-driven, i.e. the knowledge they gained is entirely from the corresponding training data. Such training mechanism may work well for traditional text genres with formal sentences; it usually achieves an unsatisfiable performance with informal text, such as social media data. Preparing substantial high-quality training data set requires a lot of manual effort
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
