Effect of emotionally toned Malay language sounds on the brain: a NIRS analysis

Abstract

Speech recognition features such as emotion have always been involved in human communication. With the recent developments in the communication methods, researchers have investigated artificial and emotional intelligence to improve communication. This has led to the emergence of affective computing, which deals with processing information pertaining to human emotions. This study aims to determine positive influence of language sounds containing emotion on brain function for improved communication. Twenty-seven college-age Japanese subjects with no prior exposure to the Malay language listened to emotionally toned and emotionally neutral sounds in the Malay language. Their brain activities were measured using near-infrared spectroscopy (NIRS) as they listened to the sounds. A comparison between different NIRS signals revealed that emotionally toned language sounds had a greater impact on brain areas associated with attention and emotion. On the contrary, emotionally neutral Malay sounds affected brain areas involved in working memory and language processing. These results suggest that emotionally-charged sounds initiate listeners’ attention and emotion recognition even when the listeners do not understand the language. The ability to interpret emotions presents challenges in computer systems and robotics; therefore, we hope that our results can be used for the development of computational models of emotion for autonomous robot research in the field of communication.