Abstract
This paper examines emotion intensity prediction in dialogs between clients and customer support representatives occurring on Twitter. We focus on a single emotion type, namely, frustration, modelling the user's level of frustration while attempting to predict frustration intensity on the current and next turn, based on the text of turns coming from both dialog participants. A subset of the Kaggle Customer Support on Twitter dataset was used as the modelling data, annotated with per-turn frustration intensity ratings. We propose to represent dialog turns by binary encoded bags of automatically selected keywords to be subsequently used in a machine learning classifier. To assess the classification quality, we examined two different levels of accuracy imprecision tolerance. Our model achieved a level of accuracy significantly higher than a statistical baseline for prediction of frustration intensity for a current turn. However, we did not find the additional information from customer support turns to help predict frustration intensity of the next turn, and the reason for that is possibly the stability of user’s frustration level over the course of the conversation, in other words, the inability of support’s response to exert much influence to user’s initial frustration level.
Highlights
With the growing popularity of social networks and the exponential increase of user-generated content volume, automated language understanding is becoming ever more relevant
Being interested in a specific task — namely, conversations between customers and customer support representatives — we concentrate on one specific emotion, frustration, and how it changes over the course of a dialog
We have presented a new dataset — a subset of the Kaggle Twitter Customer Support dialogs consisting of close to 400 dialogs and comprising almost 900 individual customer tweets, annotated for frustration intensity on the scale of 0 to 4
Summary
With the growing popularity of social networks and the exponential increase of user-generated content volume, automated language understanding is becoming ever more relevant. Humans are emotional beings, and emotions are very important for interpersonal communication For this reason, many researchers have studied automatic emotion annotation, probably for as long as the machine learning field has existed. With the development of social networks, the focus of work in emotion annotation has shifted toward emotion annotation in messages posted by users in social networks, such as Facebook, e.g. Al-Mahdawi and Teahan, 2019 [5], Weibo, e.g. Lee and Wang, 2015 [6] or Twitter, like Duppada and Hiray, 2017 [7], with Twitter being one of the most fruitful sources due to the open and concise nature of the posts it supports: short texts, sometimes accompanied by a picture or self-annotated with hashtags Such self-annotations can even be used as the foundation for gold standard corpus labelling, as done by Gonzalez-Ibanez et al in 2011 [8]. Especially in the field of business communication, automatic frustration recognition targets a relatively unaddressed need
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: International Journal of Computer Science and Information Technology
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
