Decoding Imagined Speech and Computer Control using Brain Waves

Abstract

BackgroundIn this work, we explore the possibility of decoding Imagined Speech (IS) brain waves using machine learning techniques. ApproachWe design two finite state machines to create an interface for controlling a computer system using an IS-based brain–computer interface. To decode IS signals, we propose a covariance matrix of Electroencephalogram channels as input features, covariance matrices projection to tangent space for obtaining vectors from matrices, principal component analysis for dimension reduction of vectors, an artificial neural network (ANN) as a classification model, and bootstrap aggregation for creating an ensemble of ANN models. ResultBased on these findings, we are first to use an IS-based system to operate a computer and obtain an information transfer rate of 21-bits-per-minute. The proposed approach can decode the IS signal with a mean classification accuracy of 85% on classifying one long vs. short word. Our proposed approach can also differentiate between IS and rest state brain signals with a mean classification accuracy of 94%. ComparisonAfter comparison, we show that our approach performs equivalent to the state-of-the-art approach (SOTA) on decoding long vs. short word classification task. We also show that the proposed method outperforms SOTA significantly on decoding three short words and vowels with an average margin of 11% and 9%, respectively. ConclusionThese results show that the proposed approach can decode a wide variety of IS signals and is practically applicable in a real-time environment.