Recognizing Tonal and Nontonal Mandarin Sentences for EEG-Based Brain–Computer Interface

Shiau-Ru Yang,Tzyy-Ping Jung,Chin-Teng Lin,Herming Chiueh,Li-Chun Wang,Chun-Shu Wei,Kuan-Chih Huang,Guan-Ting Liou,Yue-Loong Hsin

doi:10.1109/tcds.2021.3137251

Abstract

Most current research has focused on nontonal languages such as English. However, more than 60% of the world’s population speaks tonal languages. Mandarin is the most spoken tonal languages in the world. Interestingly, the use of tone in tonal languages may represent different meanings of words and reflect feelings, which is very different from nontonal languages. The objective of this study is to determine whether a spoken Mandarin sentence with or without tone can be distinguished by analyzing electroencephalographic (EEG) signals. We first constructed a new Brain Research Center Speech (BRCSpeech) database to recognize Mandarin. The EEG data of 14 participants were recorded, while they articulated preselected sentences. To the best of our knowledge, this is the first study to apply the method of asymmetric feature extraction method for speech recognition using EEG signals. This study shows that the feature extraction method of rational asymmetry (RASM) can achieve the best accuracy in the classification of cross-subjects. In addition, our proposed binomial variable algorithm methodology can achieve 98.82% accuracy in cross-subject classification. Furthermore, we demonstrate that the use of eight channels [(F7, F8), (C5, C6), (P5, P6), and (O1, O2)] can achieve an accurate of 94.44%. This study explores the neurophysiological correlation of Mandarin pronunciation, which can help develop a tonal language synthesis system based on BCI in the future.

Highlights

IntroductionFor patients with locked-in syndrome (LIS), such as severe spastic quadriplegic cerebral palsy, stroke and advanced amyotrophic lateral sclerosis, most of their voluntary muscles are paralyzed except for vertical eye movement communication or blink
S POKEN language is one of the most common forms of communication between people
We find that the Binomial Variable Algorithm (BVA) outperforms the Principal Component Analysis (PCA) when < 0.01 under a K-nearest neighbor (KNN) model-based classifier

Summary

Introduction

For patients with locked-in syndrome (LIS), such as severe spastic quadriplegic cerebral palsy, stroke and advanced amyotrophic lateral sclerosis, most of their voluntary muscles are paralyzed except for vertical eye movement communication or blink. Even if these patients are conscious, they cannot communicate through language. Tzyy-Ping Jung is with Swartz Center for Computational Neuroscience University of California San Diego, USA this population today. Despite motor abnormalities such as LIS or quadriplegia, their brains still function well

Objectives

Methods

Conclusion