Analysis of Linguistic and Prosodic Features of Bilingual Arabic–English Speakers for Speech Emotion Recognition

Lamiaa Abdel-Hamid,Nabil H Shaker,Ingy Emara

doi:10.1109/access.2020.2987864

Lamiaa Abdel-Hamid, Nabil H Shaker + Show 1 more

Open Access

https://doi.org/10.1109/access.2020.2987864

Copy DOI

Abstract

Speech emotion recognition (SER) research has usually focused on the analysis of the native language of speakers, most commonly, targeting European and Asian languages. In the present study, a bilingual Arabic/English speech emotion database elicited from 16 male and 16 female Egyptian participants was created in order to investigate how the linguistic and prosodic features were affected by the anger, fear, happiness and sadness emotions across Arabic and English emotional speech. The results of the linguistic analysis indicated that the participants preferred to express their emotions indirectly, mainly using religious references, and that the female participants tended to use language that was more tentative and emotionally expressive, while the male participants tended to use language that was more assertive and independent. As for the prosodic analysis, statistical t-tests showed that the prosodic features of pitch, intensity and speech rate were more indicative of anger and happiness while less relevant to fear and scarcely significant for sadness. Furthermore, speech emotion recognition performed using linear support vector machine (SVM) with AdaBoost also supported these results. In regard to first and second language linguistic features, there was no significant difference in the choice of words and structures expressing the different emotions in the two languages, but in terms of prosodic features, the females' speech showed higher pitch in Arabic in all cases while both genders showed close intensity values in the two languages and faster speech rate in Arabic than in English.

Highlights

Speech emotion recognition (SER) refers to the process of identifying human emotions from verbal speech, a matter which is done automatically by humans but is rather challenging for automatic systems [1]
Limited attention has been given to Arabic speech emotion recognition, the linguistic choice of words when expressing different basic emotions, in addition to how the implemented prosodic features differ across Arabic and English emotional speech
The aim of this study was to analyze the linguistic forms and prosodic features of emotional Arabic/English speech of native Egyptian Arabic speakers in given situations arousing the four basic emotions of anger, fear, happiness and sadness, and to investigate how they differ among the two languages for male and female participants

Summary

Introduction

Speech emotion recognition (SER) refers to the process of identifying human emotions from verbal speech, a matter which is done automatically by humans but is rather challenging for automatic systems [1]. Considering the study of SER linguistic features, Arab countries suffer from a lack of sufficient emotion recognition tools to account for the expression of emotions in the Arabic language. Previous research done in the Arab world relied mainly on manual measurements and evaluations by psychologists and linguists to account for Arab speakers’ expression of emotion through word choice [13]. Wierzbicka [2] deduced that bilingual people may express emotion differently in two different languages since emotion words in the two languages may not always match. She proposes the use of a linguistic semantic methodology known as Natural Semantic

Objectives

Results

Conclusion