Brain-computer Interaction Research Articles

A deep learning-based brain-computer interaction system for speech and motor impairment

Some people may experience accidents, strokes, or diseases that lead to both motor and speech disabilities, making it difficult to communicate with others. Those with paralysis face daily challenges in meeting their basic needs, particularly if they have difficulty speaking. Individuals with dysarthria, amyotrophic lateral sclerosis, and similar conditions may find it challenging to understand speech.The proposed system for automatic recognition of daily basic needs aims to improve the quality of life for individuals suffering from dysarthria and quadriplegic paralysis. The system achieves this by recognizing and analyzing brain signals and converting them to either audible voice commands or texts that can be sent to a healthcare provider's mobile phone based on the system settings.The proposed system uses a convolutional neural network (CNN) model to detect event-related potentials (ERPs) within the EEG signal to select one of six basic daily needs while displaying their images randomly. Ten volunteers participated in this study, contributing to the creation of the dataset used for training, testing, and validation. The proposed approach achieved an accuracy of 78.41%.

The Energy-Efficient Resource Allocation of Multi-Modal Perception for Affective Brain-Computer Interactions Based on Non-Linear Iterative Prediction Scheme

<p>For the whole environmental settings in this research, the conventional affective brain-computer interactions can not build a good performance on energy-efficient resource of network’s forwarding ports and routing paths due to its poor allocation function of cognitive radio networks, based on the novel interactive networking architecture, the model of non-linear iterative prediction scheme in interaction was successively proposed. This research proposes a modified LSTM algorithm with a structure of non-linear iterative in complexity prediction, joins the multiple k modes selection and multi-agent systems, maximizes EERA of forwarding and routing while maintaining the communication quality. Firstly, considering whether this affective brain-computer interactions need the networking communication in system. Secondly, adjusting the forwarding and routing factors of energy-efficient resource allocation by selecting the best optimal energy-efficient resource for the links through the non-linear iterative prediction in a multi-modal perception. The simulation results show that compared with the other models and algorithms, the proposed scheme for affective brain-computer interactions, which has a nice performance on a higher EERA and channel utilization of a networking architecture of brain-computer interactions.</p> <p> </p>

Recent Progress and Perspectives on Neural Chip Platforms Integrating PDMS-Based Microfluidic Devices and Microelectrode Arrays.

Recent years have witnessed a spurt of progress in the application of the encoding and decoding of neural activities to drug screening, diseases diagnosis, and brain-computer interactions. To overcome the constraints of the complexity of the brain and the ethical considerations of in vivo research, neural chip platforms integrating microfluidic devices and microelectrode arrays have been raised, which can not only customize growth paths for neurons in vitro but also monitor and modulate the specialized neural networks grown on chips. Therefore, this article reviews the developmental history of chip platforms integrating microfluidic devices and microelectrode arrays. First, we review the design and application of advanced microelectrode arrays and microfluidic devices. After, we introduce the fabrication process of neural chip platforms. Finally, we highlight the recent progress on this type of chip platform as a research tool in the field of brain science and neuroscience, focusing on neuropharmacology, neurological diseases, and simplified brain models. This is a detailed and comprehensive review of neural chip platforms. This work aims to fulfill the following three goals: (1) summarize the latest design patterns and fabrication schemes of such platforms, providing a reference for the development of other new platforms; (2) generalize several important applications of chip platforms in the field of neurology, which will attract the attention of scientists in the field; and (3) propose the developmental direction of neural chip platforms integrating microfluidic devices and microelectrode arrays.

A systematic review on automated human emotion recognition using electroencephalogram signals and artificial intelligence

Brain-Computer Interaction (BCI) system intelligence has become more dependent on electroencephalogram (EEG)-based emotion recognition because of the numerous applications of emotion classification, such as recommender systems, cognitive load detection, etc. Emotion classification has drawn the recent buzz in Artificial Intelligence (AI)-powered research. In this article, we presented a systematic review of automated emotion recognition from EEG signals using AI. The review process is carried out based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses(PRISMA). After that EEG databases, and EEG preprocessing methods are included in this study. Also included feature extraction and feature selection methods. In addition, the included studies were divided into two types: i)deep learning(DL)-based emotion identification systems and ii) machine learning(ML)-based emotion classification models. The examined systems are analyzed based on their features, classification methodologies, classifiers, types of classified emotions, accuracy, and the datasets they employed. There is also an interesting comparison, a look at feature research trends, and ideas for new areas to study.

A 2D CNN-LSTM hybrid algorithm using time series segments of EEG data for motor imagery classification

Motor imagery-based brain–computer interaction (MI-BCI) converts human neural activity into computational information, often used as commands, by recognizing electroencephalogram (EEG) patterns in different MI tasks. Owing to the poor accuracy and stability of classification algorithms, MI systems remain challenging to employ in practice. Therefore, we proposed a two-dimensional convolutional neural network-long short-term memory (2D CNN-LSTM) hybrid algorithm to classify EEG in MI tasks. We converted EEG signals into time series segments, and then extracted the connectivity features between the different EEG channels in each segment through 2D CNN, finally sent the feature vectors to LSTM network for training. The experimental results showed that the 2D CNN-LSTM algorithm had an average accuracy of 98.5% in the validation set, and a 93.3% average accuracy in the test set, and compared with existing algorithm, such as 1D CNN-LSTM algorithm in the test set, the average accuracy of our algorithm was improved by 3.1% and the F1-score by 0.04, while the loss of the verification set decreased by more than 0.1 in multi-person modeling (general model) in the test set, which was the largest decrease among the algorithms being compared.

Emotion recognition using spatial-temporal EEG features through convolutional graph attention network

Objective. Constructing an efficient human emotion recognition model based on electroencephalogram (EEG) signals is significant for realizing emotional brain–computer interaction and improving machine intelligence. Approach. In this paper, we present a spatial-temporal feature fused convolutional graph attention network (STFCGAT) model based on multi-channel EEG signals for human emotion recognition. First, we combined the single-channel differential entropy (DE) feature with the cross-channel functional connectivity (FC) feature to extract both the temporal variation and spatial topological information of EEG. After that, a novel convolutional graph attention network was used to fuse the DE and FC features and further extract higher-level graph structural information with sufficient expressive power for emotion recognition. Furthermore, we introduced a multi-headed attention mechanism in graph neural networks to improve the generalization ability of the model. Main results. We evaluated the emotion recognition performance of our proposed model on the public SEED and DEAP datasets, which achieved a classification accuracy of 99.11% ± 0.83% and 94.83% ± 3.41% in the subject-dependent and subject-independent experiments on the SEED dataset, and achieved an accuracy of 91.19% ± 1.24% and 92.03% ± 4.57% for discrimination of arousal and valence in subject-independent experiments on DEAP dataset. Notably, our model achieved state-of-the-art performance on cross-subject emotion recognition tasks for both datasets. In addition, we gained insight into the proposed frame through both the ablation experiments and the analysis of spatial patterns of FC and DE features. Significance. All these results prove the effectiveness of the STFCGAT architecture for emotion recognition and also indicate that there are significant differences in the spatial-temporal characteristics of the brain under different emotional states.

РАЗРАБОТКА НЕЙРОСЕТЕВЫХ ПРИЛОЖЕНИЙ ВЗАИМОДЕЙСТВИЯ МОЗГ-КОМПЬЮТЕР НА ОСНОВЕ МОДЕЛИ СТАБИЛЬНОЙ ДИФФУЗИИ

This paper investigates neurotechnologies for developing braincomputer interaction (BCI) based on the generative deep learning Stable Diffusion model. An algorithm for modeling BCI is proposed, its training and testing on artificial data are described. The results are encouraging for researchers and can be used in various areas of BCI, such as distance learning, remote medicine, etc.

EEG-Based Emotion Recognition Using Spatial-Temporal-Connective Features via Multi-Scale CNN

Electroencephalography (EEG) is often used for emotion recognition in brain computer interaction (BCI) research. EEG signals from each channel mainly reflect activities of the brain region close to the channel position, and the activities cooperated by various brain regions are response to the emotion-induced stimuli. In this paper, temporal, spatial and connective features are extracted from EEG signals gotten around the head, and used for emotion recognition via a proposed model, spatial-temporal-connective convolutional neural network (STC-CNN). The channel-to-channel connectivity is gotten to describe brain region-to-region cooperation under emotion stimuli. The STC-CNN achieved an average accuracy of 96.79 percent and 96.89 percent after classifying in two emotion dimensions of arousal and valence. Utilization of the method we proposed not only helped to achieve a higher accuracy of emotion recognition, but also explored a new selection of feature for EEG research.

Brain-computer interaction neurointerface based on artificial intelligence and its parallel programming using high-performance calculation on cluster mobile devices

The paper deals with hardware and software support for the interaction of human brain activity with the dynamic movement of the part of its upper limb based on artificial intelligence and its parallel programming using high-performance computer calculation on cluster mobile devices. The obtained results can be used as a basis for the development of high-performance software and hardware for the effective operation of brain-computer interaction neuro interfaces.

Enhanced Adaptive Brain-Computer Interface Approach for Intelligent Assistance to Disabled Peoples

Assistive devices for disabled people with the help of Brain-Computer Interaction (BCI) technology are becoming vital bio-medical engineering. People with physical disabilities need some assistive devices to perform their daily tasks. In these devices, higher latency factors need to be addressed appropriately. Therefore, the main goal of this research is to implement a real-time BCI architecture with minimum latency for command actuation. The proposed architecture is capable to communicate between different modules of the system by adopting an automotive, intelligent data processing and classification approach. Neuro-sky mind wave device has been used to transfer the data to our implemented server for command propulsion. Think-Net Convolutional Neural Network (TN-CNN) architecture has been proposed to recognize the brain signals and classify them into six primary mental states for data classification. Data collection and processing are the responsibility of the central integrated server for system load minimization. Testing of implemented architecture and deep learning model shows excellent results. The proposed system integrity level was the minimum data loss and the accurate commands processing mechanism. The training and testing results are 99% and 93% for custom model implementation based on TN-CNN. The proposed real-time architecture is capable of intelligent data processing unit with fewer errors, and it will benefit assistive devices working on the local server and cloud server.

Hair-compatible sponge electrodes integrated on VR headset for electroencephalography

Virtual reality (VR) technology has emerged as a promising tool for brain-computer interaction and neuroscience research due to its ability to provide immersive and interactive experiences for its users. As a powerful tool to noninvasively monitor the cortex, electroencephalography (EEG) combined with VR represents an exciting opportunity for the measurement of brain activity during these experiences, providing insight into cognitive and neural processes. However, traditional gel-based EEG sensors are not compatible with VR headsets, and most emerging VR-EEG headsets utilizing rigid comb electrodes are uncomfortable after prolonged wear. To address this limitation, we created soft, porous, and hair-compatible sponge electrodes based on conductive poly(3,4-ethylenedioxythiophene) polystyrene sulfonate/melamine (PMA) and integrated them onto a VR headset through a customized, flexible circuit for multichannel EEG during VR task performing. Our PMA sponge electrodes can deform to make contact with the scalp skin through hairs under the pressure naturally applied by the strap of the VR headset. The specific contact impedance was consistently below 80 kΩ·cm2, even at hairy sites. We demonstrated the capability of our VR-EEG headset by recording alpha rhythms during eye closure at both hairless and hairy sites. In another demonstration, we developed a VR task to evoke the contingent negative variation potential and achieved a classification accuracy of 0.66 ± 0.07, represented by the cross-validated area under the receiver operating characteristic curve. Our sponge-electrode-integrated VR headset is user-friendly and easy to set up, marking a step toward future reliable, comfortable, and reusable VR-EEG technology.

A Review of Brain Activity and EEG-Based Brain-Computer Interfaces for Rehabilitation Application.

Patients with severe CNS injuries struggle primarily with their sensorimotor function and communication with the outside world. There is an urgent need for advanced neural rehabilitation and intelligent interaction technology to provide help for patients with nerve injuries. Recent studies have established the brain-computer interface (BCI) in order to provide patients with appropriate interaction methods or more intelligent rehabilitation training. This paper reviews the most recent research on brain-computer-interface-based non-invasive rehabilitation systems. Various endogenous and exogenous methods, advantages, limitations, and challenges are discussed and proposed. In addition, the paper discusses the communication between the various brain-computer interface modes used between severely paralyzed and locked patients and the surrounding environment, particularly the brain-computer interaction system utilizing exogenous (induced) EEG signals (such as P300 and SSVEP). This discussion reveals with an examination of the interface for collecting EEG signals, EEG components, and signal postprocessing. Furthermore, the paper describes the development of natural interaction strategies, with a focus on signal acquisition, data processing, pattern recognition algorithms, and control techniques.

Study of event-related potentials by withdrawal friction on the fingertip.

The lack of understanding about the brain's reaction processes in perceiving touch and separation between skin and object surfaces is a barrier to the development of existing brain-computer interface technologies and virtual haptics. These technologies are limited in their ability to advance. It leaves prosthesis users with a limited amount of tactile information that they can feel. This study aims to determine whether distinct surface aspects of various items trigger different reactions from the brain when friction is removed from the surface. When friction is suddenly removed from the surface of an item, a technique called event-related potential, (ERP) is used to study the features of people's EEGs. It is done after the subject has actively explored the object's surface. A 64-channels EEG collecting system was utilized to acquire EEG data from the individuals. [Corrections added on 5 December 2022, after first online publication: The preceding sentence has been updated.] The event-related potentials for friction removal were generated using the Oddball paradigm, and the samples consisted of sandpaper with three distinct degrees of roughness. We utilized a total of 20 participants, 10 of whom were male, and 10 of whom were female, with a mean age of 21 years. It was discovered that the P3 component of event-related potentials, which is essential for cognition, was noticeably absent in the friction withdrawal response for various roughnesses. It was the case regardless of whether the surface was smooth or rough. Moreover, there was no statistically significant difference between the P1 andP2 components, which suggests that the brain could not recognize the surface properties of objects with varying roughness as the friction withdrawal was being performed. It has been demonstrated that tactile recognition does not occur after friction withdrawal. The findings of this paper could have significant repercussions for future research involving the study of haptic perception and brain-computer interaction in prosthetic hands. It is a step toward future research on the mechanisms underlying human tactile perception, so think of it as preparation.

Exploration of Brain-Computer Interaction for Supporting Children's Attention Training: A Multimodal Design Based on Attention Network and Gamification Design.

Recent developments in brain-computer interface (BCI) technology have shown great potential in terms of estimating users' mental state and supporting children's attention training. However, existing training tasks are relatively simple and lack a reliable task-generation process. Moreover, the training experience has not been deeply studied, and the empirical validation of the training effect is still insufficient. This study thusly proposed a BCI training system for children's attention improvement. In particular, to achieve a systematic training process, the attention network was referred to generate the training games for alerting, orienting and executive attentions, and to improve the training experience and adherence, the gamification design theory was introduced to derive attractive training tasks. A preliminary experiment was conducted to set and modify the training parameters. Subsequently, a series of contrasting user experiments were organized to examine the impact of BCI training. To test the training effect of the proposed system, a hypothesis-testing approach was adopted. The results revealed that the proposed BCI gamification attention training system can significantly improve the participants' attention behaviors and concentration ability. Moreover, an immersive, inspiring and smooth training process can be created, and a pleasant user experience can be achieved. Generally, this work is promising in terms of providing a valuable reference for related practices, especially for how to generate BCI attention training tasks using attention networks and how to improve training adherence by integrating multimodal gamification elements.

Decoding lexical tones and vowels in imagined tonal monosyllables using fNIRS signals

Objective. Speech is a common way of communication. Decoding verbal intent could provide a naturalistic communication way for people with severe motor disabilities. Active brain computer interaction (BCI) speller is one of the most commonly used speech BCIs. To reduce the spelling time of Chinese words, identifying vowels and tones that are embedded in imagined Chinese words is essential. Functional near-infrared spectroscopy (fNIRS) has been widely used in BCI because it is portable, non-invasive, safe, low cost, and has a relatively high spatial resolution. Approach. In this study, an active BCI speller based on fNIRS is presented by covertly rehearsing tonal monosyllables with vowels (i.e. /a/, /i/, /o/, and /u/) and four lexical tones in Mandarin Chinese (i.e. tones 1, 2, 3, and 4) for 10 s. Main results. fNIRS results showed significant differences in the right superior temporal gyrus between imagined vowels with tone 2/3/4 and those with tone 1 (i.e. more activations and stronger connections to other brain regions for imagined vowels with tones 2/3/4 than for those with tone 1). Speech-related areas for tone imagery (i.e. the right hemisphere) provided majority of information for identifying tones, while the left hemisphere had advantages in vowel identification. Having decoded both vowels and tones during the post-stimulus 15 s period, the average classification accuracies exceeded 40% and 70% in multiclass (i.e. four classes) and binary settings, respectively. To spell words more quickly, the time window size for decoding was reduced from 15 s to 2.5 s while the classification accuracies were not significantly reduced. Significance. For the first time, this work demonstrated the possibility of discriminating lexical tones and vowels in imagined tonal syllables simultaneously. In addition, the reduced time window for decoding indicated that the spelling time of Chinese words could be significantly reduced in the fNIRS-based BCIs.

Exploring the Visual Guidance of Motor Imagery in Sustainable Brain–Computer Interfaces

Motor imagery brain–computer interface (MI-BCI) systems hold the possibility of restoring motor function and also offer the possibility of sustainable autonomous living for individuals with various motor and sensory impairments. When utilizing the MI-BCI, the user’s performance impacts the system’s overall accuracy, and concentrating on the user’s mental load enables a better evaluation of the system’s overall performance. The impacts of various levels of abstraction on visual guidance of mental training in motor imagery (MI) may be comprehended. We proposed hypotheses about the effects of visually guided abstraction on brain activity, mental load, and MI-BCI performance, then used the event-related desynchronization (ERD) value to measure the user’s brain activity, extracted the brain power spectral density (PSD) to measure the brain load, and finally classified the left- and right-handed MI through a support vector machine (SVM) classifier. The results showed that visual guidance with a low level of abstraction could help users to achieve the highest brain activity and the lowest mental load, and the highest accuracy rate of MI classification was 97.14%. The findings imply that to improve brain–computer interaction and enable those less capable to regain their mobility, visual guidance with a low level of abstraction should be employed when training brain–computer interface users. We anticipate that the results of this study will have considerable implications for human-computer interaction research in BCI.

A vision towards integrated 6G communication networks: Promising technologies, architecture, and use-cases

The evolution of the previous mobile communication generations has led to innovative goals of the Internet of Everything (IoE) in the 5G. However, addressing all IoE-associated problems in 5G is difficult and a long-term process. As the key performance indicators (KPIs) of the 5G services are highly diverse, it is an intimidating task to develop a single platform enabling all KPIs. The vision of next-generation 6G wireless communications lies not only in enhancing these targets but also in providing new services. Numerous extensively envisaged future services, including life-critical services and wireless brain–computer interactions, will be critically dependent on an instant, virtually unlimited wireless connectivity. In this direction, the 6G is envisioned to have primely five service objectives; further-enhanced mobile broadband (FeMBB), ultra-massive machine type communication (umMTC), extremely reliable low latency communication (ERLLC), long-distance and high-mobility communications (LDHMC), and extremely low-power communications (ELPC). The 3D global integration of the wireless communication networks is lacking in the 5G, which is targeted by the future 6G. In this paper, we present an exhaustive review of the 6G wireless communication network. We explore the various existing mobile communication generations concerning data rate, frequency band, bandwidth allotted, latency, and applications. We also highlight various current trends and issues in the 5G communication network, which drives research for the 6G communication network. Our focus is to provide a comprehensive survey on the future 6G. So, we explored the objectives and design principles for 6G. This paper highlights the key 6G technology drivers. This paper also proposes an architectural design for 6G. Moreover, we carry out a case-study of 6G architecture operational design and compare the result with previous generation architecture designs. Further, 6G envisioned open research challenges, research directions, and recent advancements are also highlighted in this paper. Furthermore, we discuss possible use-cases in terms of real-time interactions of the biological, physical, and digital world, and also how these use-cases are going to serve in 6G.

Multimodal Opera Performance Form Based on Human-Computer Interaction Technology

“Audience Engagement (AE)” describes how a stage performance affects the audience’s thoughts, provokes a bodily response, and spurs cognitive growth. With little audience involvement, theatre performing arts like opera typically have difficulty keeping audiences’ attention. The brain-computer interaction (BCI) technology could be used in opera performances to alter the audience’s emotional experience. Nevertheless, for such BCI systems to function, they must accurately identify their participants’ present emotional states. Although difficult to evaluate, audience participation is a vital sign of how well an opera performs. Practical methodological approaches for real-time perception and comprehension of audience emotions include psychological and physiological assessments. Hence, a multimodal emotional state detection technique (MESDT) for enhancing the AE in opera performance using BCI has been proposed. Three essential steps make up a conceptual MESDT architecture. An electroencephalogram (EEG) and other biological signs from the audience are first captured. Second, the acquired signals are processed, and the BCI tries to determine the user’s present psychological response. Third, an adaptive performance stimulus (APS) is triggered to enhance AE in opera performance, as determined by a rule base. To give the opera audience a high-quality viewing experience, the immersive theatre performance has been simulated. Fifty individuals have been used in the experimental assessment and performance studies. The findings demonstrated that the proposed technology had been able to accurately identify the decline in AE and that performing stimuli had a good impact on enhancing AE during an opera performance. It has been further shown that the suggested design improves the overall performance of AE by 5.8% when compared to a typical BCI design (one that uses EEG characteristics solely) for the proposed MESDT framework with BCI.

Enhancing psychosomatic health using artificial intelligence-based treatment protocol: A data science-driven approach

The present study opens an avenue for an improved holistic state of health. Several parameters related to an individual's cognitive interactions to manage stress have been explored. It induces multiple therapeutic interventions in an experimental setting to fulfill the same. This is a quantitative study in which relevant data are captured using a questionnaire and Brain-Computer Interaction (BCI) simultaneously. The research findings establish the role of significant factors that help to determine psychosomatic health.Data-driven models built using unique cognitive instances can augment psychosomatic health in terms of time, cost, and access. Since this study is based on a technological foundation defined by an electroencephalograph (EEG), it can further be scaled for other healthcare applications and incorporated into ‘Digital Twin.’ Using Artificial Intelligence (AI) based treatment protocol, the findings could be extended to the healthcare ecosystem.

A data augmentation and channel selection technique for grading human emotions on DEAP dataset

Emotion Recognition is one of the most important research area in the domain of Brain Computer Interactions. Human activities are influenced by emotions. Emotion recognition is carried out using gesture recognition, facial expressions etc. These methods are inconvenient and require quick feedback from users. Recently, Electroencephalogram technology has been found to be very efficient for emotion recognition task. Multi-Channel EEG headset is found to be an effective technology for BCI. However, it generates huge channel data and data obtained from many channels do not play effective role in identification of emotional state.In this paper, we use publicly available DEAP dataset as a source of EEG signals. Two major issues regrading the EEG data analysis is being addressed in this research. The first issue is the availability of the small number of samples. To address this issue, we exploit signal processing techniques. Multi-channel EEG with a large sampling frequency produces huge data per channel. All the channels are not important for emotion analysis and hence to address this second issue we explore metaheuristic algorithms to obtain an optimal subset of channels for emotion classification. Obtained results are very promising with 92.5% and 81.25% for two class classification in valence and arousal emotions and our proposed work can be used for practical applications of emotion classification.