Emotion Detection for Social Robots Based on NLP Transformers and an Emotion Ontology.

Wilfredo Graterol,Cleia Santos-Libarino,Yudith Cardinale,Irvin Dongo,Edmundo Lopes-Silva,Jose Diaz-Amado

doi:10.3390/s21041322

Abstract

For social robots, knowledge regarding human emotional states is an essential part of adapting their behavior or associating emotions to other entities. Robots gather the information from which emotion detection is processed via different media, such as text, speech, images, or videos. The multimedia content is then properly processed to recognize emotions/sentiments, for example, by analyzing faces and postures in images/videos based on machine learning techniques or by converting speech into text to perform emotion detection with natural language processing (NLP) techniques. Keeping this information in semantic repositories offers a wide range of possibilities for implementing smart applications. We propose a framework to allow social robots to detect emotions and to store this information in a semantic repository, based on EMONTO (an EMotion ONTOlogy), and in the first figure or table caption. Please define if appropriate. an ontology to represent emotions. As a proof-of-concept, we develop a first version of this framework focused on emotion detection in text, which can be obtained directly as text or by converting speech to text. We tested the implementation with a case study of tour-guide robots for museums that rely on a speech-to-text converter based on the Google Application Programming Interface (API) and a Python library, a neural network to label the emotions in texts based on NLP transformers, and EMONTO integrated with an ontology for museums; thus, it is possible to register the emotions that artworks produce in visitors. We evaluate the classification model, obtaining equivalent results compared with a state-of-the-art transformer-based model and with a clear roadmap for improvement.

Highlights

IntroductionThe presence of robots are becoming more common in human daily life [1]. Service robots concretely share environments with human beings to actively collaborate with them in specific daily tasks, such as serving as an assistant to nurses in patient walking and patient sitting tasks in hospital environments [2] or working as a student receptionist at a university [3]
Nowadays, the presence of robots are becoming more common in human daily life [1]
We propose EMONTO, an extensible ontology of emotions, and we incorporate it into the framework; this semantic information can be used for many proposes in the context of social robotics

Summary

Introduction

The presence of robots are becoming more common in human daily life [1]. Service robots concretely share environments with human beings to actively collaborate with them in specific daily tasks, such as serving as an assistant to nurses in patient walking and patient sitting tasks in hospital environments [2] or working as a student receptionist at a university [3]. The detected emotion can be used for many purposes, such as dictating a robot’s behavior and adapting interactions with humans (in real-time) [4,5,6,7,8] or associating the emotion to events or objects in specific domains (e.g., opinions about commercial products, emotions produced by artworks in museums, and emotions produced by plates in restaurants) for further analysis [9,10] From this perspective, keeping this information in semantic repositories offers a wide range of possibilities for automatically modeling the robot’s behavior or for supporting the implementation of smart applications

Objectives

Findings

Discussion

Conclusion