Emotion Estimation Method Based on Emoticon Image Features and Distributed Representations of Sentences

Akira Fujisawa,Kenji Kita,Minoru Yoshida,Kazuyuki Matsumoto

doi:10.3390/app12031256

Akira Fujisawa, Kenji Kita + Show 2 more

Open Access

https://doi.org/10.3390/app12031256

Copy DOI

Journal: Applied sciences	Publication Date: Jan 25, 2022
Citations: 2	License type: CC BY 4.0

Affiliation: Aomori University, Tokushima University

Abstract

This paper proposes an emotion recognition method for tweets containing emoticons using their emoticon image and language features. Some of the existing methods register emoticons and their facial expression categories in a dictionary and use them, while other methods recognize emoticon facial expressions based on the various elements of the emoticons. However, highly accurate emotion recognition cannot be performed unless the recognition is based on a combination of the features of sentences and emoticons. Therefore, we propose a model that recognizes emotions by extracting the shape features of emoticons from their image data and applying the feature vector input that combines the image features with features extracted from the text of the tweets. Based on evaluation experiments, the proposed method is confirmed to achieve high accuracy and shown to be more effective than methods that use text features only.

Highlights

Emoticons are nonverbal expressions that are composed by combining characters
To ascertain the emotions expressed by entire sentences from tweets including emoticons, we focus on the shape features of the emoticons and propose a method to combine these extracted features with textual features
We proposed a more accurate method than existing approaches for estimating emotions from tweets containing emoticons by extracting features from the images of the emoticons, in addition to distributed expressions extracted from the text of the tweets

Summary

Introduction

Emoticons are nonverbal expressions that are composed by combining characters. They are typically embedded in text and are widely used in multibyte character languages that have many character types, such as Japanese. By adding emoticons to text, it is possible to flexibly express emotions and intentions that are otherwise difficult to convey using only character information. In addition to emoticons, various other forms of nonverbal expressions, such as emojis and stamps, are used in text-based communications. It is difficult to ascertain emotions and intentions using only emoticons. To ascertain the emotions expressed by entire sentences from tweets including emoticons, we focus on the shape features of the emoticons and propose a method to combine these extracted features with textual features

Methods

Results

Discussion

Conclusion