Decoding of Chinese phoneme clusters using ECoG

Abstract

A finite set of phonetic units is used in human speech, but how our brain recognizes these units from speech streams is still largely unknown. The revealing of this neural mechanism may lead to the development of new types of speech brain computer interfaces (BCI) and computer speech recognition systems. In this study, we used electrocorticography (ECoG) signal from human cortex to decode phonetic units during the perception of continuous speech. By exploring the wavelet time-frequency features, we identified ECoG electrodes that have selective response to specific Chinese phonemes. Gamma and high-gamma power of these electrodes were further combined to separate sets of phonemes into clusters. The clustered organization largely coincided with phonological categories defined by the place of articulation and manner of articulation. These findings were incorporated into a decoding framework of Chinese phonemes clusters. Using support vector machine (SVM) classifier, we achieved consistent accuracies higher than chance level across five patients discriminating specific phonetic clusters, which suggests a promising direction of implementing a speech BCI.