Speech Emotion Recognition: Humans vs Machines

S Werner,G N Petrenko

doi:10.32603/2412-8562-2019-5-5-136-152

S Werner, G N Petrenko

Open Access

https://doi.org/10.32603/2412-8562-2019-5-5-136-152

Copy DOI

Abstract

Introduction.The study focuses on emotional speech perception and speech emotion recognition using prosodic clues alone. Theoretical problems of defining prosody, intonation and emotion along with the challenges of emotion classification are discussed. An overview of acoustic and perceptional correlates of emotions found in speech is provided. Technical approaches to speech emotion recognition are also considered in the light of the latest emotional speech automatic classification experiments.Methodology and sources.The typical “big six” classification commonly used in technical applications is chosen and modified to include such emotions as disgust and shame. A database of emotional speech in Russian is created under sound laboratory conditions. A perception experiment is run using Praat software’s experimental environment.Results and discussion. Cross-cultural emotion recognition possibilities are revealed, as the Finnish and international participants recognised about a half of samples correctly. Nonetheless, native speakers of Russian appear to distinguish a larger proportion of emotions correctly. The effects of foreign languages knowledge, musical training and gender on the performance in the experiment were insufficiently prominent. The most commonly confused pairs of emotions, such as shame and sadness, surprise and fear, anger and disgust as well as confusions with neutral emotion were also given due attention.Conclusion.The work can contribute to psychological studies, clarifying emotion classification and gender aspect of emotionality, linguistic research, providing new evidence for prosodic and comparative language studies, and language technology, deepening the understanding of possible challenges for SER systems.

Highlights

The study focuses on emotional speech perception and speech emotion recognition using prosodic clues alone
Technical approaches to speech emotion recognition are considered in the light of the latest emotional speech automatic classification experiments
That is why we can make a positive conclusion regrading the possibility of cross-cultural speech emotion recognition

Summary

Introduction

The study focuses on emotional speech perception and speech emotion recognition using prosodic clues alone. Theoretical problems of defining prosody, intonation and emotion along with the challenges of emotion classification are discussed. An overview of acoustic and perceptional correlates of emotions found in speech is provided. Technical approaches to speech emotion recognition are considered in the light of the latest emotional speech automatic classification experiments. The work can contribute to psychological studies, clarifying emotion classification and gender aspect of emotionality, linguistic research, providing new evidence for prosodic and comparative language studies, and language technology, deepening the understanding of possible challenges for SER systems. The study aims to bring the existing variety of approaches together and try to answer some of the most topical questions, such as what features are informative when differentiating between emotions, whether humans or machines are better at recognising emotions in speech, and if the emotions in speech are cross-culturally recognisable. A much bigger portion of meaning comes non-verbally and paraverbally, the later term comprising prosodic (intonational) phenomena

Objectives

Methods

Results

Conclusion