Brain-machine Interface System Research Articles

A Brain-Controlled and User-Centered Intelligent Wheelchair: A Feasibility Study.

Recently, due to physical aging, diseases, accidents, and other factors, the population with lower limb disabilities has been increasing, and there is consequently a growing demand for wheelchair products. Modern product design tends to be more intelligent and multi-functional than in the past, with the popularization of intelligent concepts. This supports the design of a new, fully functional, intelligent wheelchair that can assist people with lower limb disabilities in their day-to-day life. Based on the UCD (user-centered design) concept, this study focused on the needs of people with lower limb disabilities. Accordingly, the demand for different functions of intelligent wheelchair products was studied through a questionnaire survey, interview survey, literature review, expert consultation, etc., and the function and appearance of the intelligent wheelchair were then defined. A brain-machine interface system was developed for controlling the motion of the intelligent wheelchair, catering to the needs of disabled individuals. Furthermore, ergonomics theory was used as a guide to determine the size of the intelligent wheelchair seat, and eventually, a new intelligent wheelchair with the features of climbing stairs, posture adjustment, seat elevation, easy interaction, etc., was developed. This paper provides a reference for the design upgrade of the subsequently developed intelligent wheelchair products.

A wireless neural recording microsystem with operator-based spike detection

We introduce an innovative approach that incorporates operator-based spike detection in wireless microsystems for neural signal processing. Through comparative analyses between simple thresholding and operator-based detection conducted on pre-recorded spike detection experiments, our research emphasizes the superiority of the operator-based spike detection approach. The operator-based spike detection emerges as a promising technique for miniaturized wireless neural signal devices, primarily due to its proficient noise-handling capabilities paired with reduced power consumption. Furthermore, its adaptability across various experimental conditions amplifies its versatility. Empirical tests underscored its low power requisites and compactness, emphasizing practical utility of the detection scheme in the neural microsystems. Collectively, our results mark a significant progression in wireless cerebral signal recording methodologies, paving the way for optimized wireless brain-machine interface (BMI) systems.

How Integration of a Brain-Machine Interface and Obstacle Detection System Can Improve Wheelchair Control via Movement Imagery.

This study presents a human-computer interaction combined with a brain-machine interface (BMI) and obstacle detection system for remote control of a wheeled robot through movement imagery, providing a potential solution for individuals facing challenges with conventional vehicle operation. The primary focus of this work is the classification of surface EEG signals related to mental activity when envisioning movement and deep relaxation states. Additionally, this work presents a system for obstacle detection based on image processing. The implemented system constitutes a complementary part of the interface. The main contributions of this work include the proposal of a modified 10-20-electrode setup suitable for motor imagery classification, the design of two convolutional neural network (CNNs) models employed to classify signals acquired from sixteen EEG channels, and the implementation of an obstacle detection system based on computer vision integrated with a brain-machine interface. The models developed in this study achieved an accuracy of 83% in classifying EEG signals. The resulting classification outcomes were subsequently utilized to control the movement of a mobile robot. Experimental trials conducted on a designated test track demonstrated real-time control of the robot. The findings indicate the feasibility of integration of the obstacle detection system for collision avoidance with the classification of motor imagery for the purpose of brain-machine interface control of vehicles. The elaborated solution could help paralyzed patients to safely control a wheelchair through EEG and effectively prevent unintended vehicle movements.

Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

Effectiveness of the Combined Use of a Brain-Machine Interface System and Virtual Reality as a Therapeutic Approach in Patients with Spinal Cord Injury: A Systematic Review.

Spinal cord injury has a major impact on both the individual and society. This damage can cause permanent loss of sensorimotor functions, leading to structural and functional changes in somatotopic regions of the spinal cord. The combined use of a brain-machine interface and virtual reality offers a therapeutic alternative to be considered in the treatment of this pathology. This systematic review aimed to evaluate the effectiveness of the combined use of virtual reality and the brain-machine interface in the treatment of spinal cord injuries. A search was performed in PubMed, Web of Science, PEDro, Cochrane Central Register of Controlled Trials, CINAHL, Scopus, and Medline, including articles published from the beginning of each database until January 2023. Articles were selected based on strict inclusion and exclusion criteria. The Cochrane Collaboration's tool was used to assess the risk of bias and the PEDro scale and SCIRE systems were used to evaluate the methodological quality of the studies. Eleven articles were selected from a total of eighty-two. Statistically significant changes were found in the upper limb, involving improvements in shoulder and upper arm mobility, and weaker muscles were strengthened. In conclusion, most of the articles analyzed used the electroencephalogram as a measurement instrument for the assessment of various parameters, and most studies have shown improvements. Nonetheless, further research is needed with a larger sample size and long-term follow-up to establish conclusive results regarding the effect size of these interventions.

Differential expression of genes involved in the chronic response to intracortical microelectrodes.

Brain-Machine Interface systems (BMIs) are clinically valuable devices that can provide functional restoration for patients with spinal cord injury or improved integration for patients requiring prostheses. Intracortical microelectrodes can record neuronal action potentials at a resolution necessary for precisely controlling BMIs. However, intracortical microelectrodes have a demonstrated history of progressive decline in the recording performance with time, inhibiting their usefulness. One major contributor to decreased performance is the neuroinflammatory response to the implanted microelectrodes. The neuroinflammatory response can lead to neurodegeneration and the formation of a glial scar at the implant site. Historically, histological imaging of relatively few known cellular and protein markers has characterized the neuroinflammatory response to implanted microelectrode arrays. However, neuroinflammation requires many molecular players to coordinate the response - meaning traditional methods could result in an incomplete understanding. Taking advantage of recent advancements in tools to characterize the relative or absolute DNA/RNA expression levels, a few groups have begun to explore gene expression at the microelectrode-tissue interface. We have utilized a custom panel of ∼813 neuroinflammatory-specific genes developed with NanoString for bulk tissue analysis at the microelectrode-tissue interface. Our previous studies characterized the acute innate immune response to intracortical microelectrodes. Here we investigated the gene expression at the microelectrode-tissue interface in wild-type (WT) mice chronically implanted with nonfunctioning probes. We found 28 differentially expressed genes at chronic time points (4WK, 8WK, and 16WK), many in the complement and extracellular matrix system. Further, the expression levels were relatively stable over time. Genes identified here represent chronic molecular players at the microelectrode implant sites and potential therapeutic targets for the long-term integration of microelectrodes. STATEMENT OF SIGNIFICANCE: Intracortical microelectrodes can record neuronal action potentials at a resolution necessary for the precise control of Brain-Machine Interface systems (BMIs). However, intracortical microelectrodes have a demonstrated history of progressive declines in the recording performance with time, inhibiting their usefulness. One major contributor to the decline in these devices is the neuroinflammatory response against the implanted microelectrodes. Historically, neuroinflammation to implanted microelectrode arrays has been characterized by histological imaging of relatively few known cellular and protein markers. Few studies have begun to develop a more in-depth understanding of the molecular pathways facilitating device-mediated neuroinflammation. Here, we are among the first to identify genetic pathways that could represent targets to improve the host response to intracortical microelectrodes, and ultimately device performance.

Biomedical Amplifiers Design Based on Pseudo-Resistors: A Review

The demand for efficient, robust, and cost-effective Brain-Machine Interface (BMI) systems continues to increase in the last decade. One of the fundamental design blocks in such systems is the signal amplification and filtering. Generally, biomedical signals are characterized by low amplitude and low frequency in a noisy environment. Therefore they need to be amplified and filtered before passing the signal to the next processing stage. In this review paper, a comprehensive survey is conducted in existing literature of two-stage biomedical amplifiers, focusing on the impact of the pseudo-resistor non-linearity on the system’s performance. First, the common categories of pseudo-resistors are presented and discussed. Then, different amplifier designs, targeted for biomedical applications, are identified and studied considering the influence of the pseudo-resistors on the performance. A special focus was given to the impact of the Process, Voltage, and Temperature variations where experiments are conducted to test the performance under different variation tests. Different two-stage biomedical amplifiers, used in bio-detection systems, with programmable gain and bandwidth features based on pseudo-resistors are implemented. The designs are realized and simulated using LTspice utilizing 90nm process technology, BSIM4, version 4.3, level 54.

ID: 222352 Vasospasm Secondary to Responsive Neurostimulator Placement: A Previously Unreported Complication

The Responsive Neurostimulation (RNS) system is a therapy for patients with drug-resistant epilepsy who are not candidates for surgical resection of a seizure focus. The system involves long-term implantation of subdural or depth lead connected to a neurostimulator that detects incipient epileptic activity and delivers stimulation to control seizure spread. The intra-cranial leads are implanted as close as possible to the epileptic focus or foci to optimize detection and therapeutic stimulation ability. As with other brain-machine interface systems, RNS systems are subject to hardware-related complications. As a relatively new therapy option, the adverse effects of long-term implantation are still being clarified. We present a case report of a patient that developed severe vasospasm after RNS implantation.

Empirical comparison of deep learning methods for EEG decoding.

Electroencephalography (EEG) is a technique that can be used in non-invasive brain-machine interface (BMI) systems to register brain electrical activity. The EEG signals are non-linear and non-stationary, making the decoding procedure a complex task. Deep learning techniques have been successfully applied in several research fields, often improving the results compared with traditional approaches. Therefore, it is believed that these techniques can also improve the process of decoding brain signals in BMI systems. In this work, we present the implementation of two deep learning-based decoders and we compared the results with other state of art deep learning methods. The first decoder uses long short-term memory (LSTM) recurrent neural network and the second, entitled EEGNet-LSTM, combines a well-known neural decoder based on convolutional neural networks, called EEGNet, with some LSTM layers. The decoders have been tested using data set 2a from BCI Competition IV, and the results showed that the EEGNet-LSTM decoder has been approximately 23% better than the competition-winning decoder. A Wilcoxon t-test showed a significant difference between the two decoders (Z = 2.524, p = 0.012). The LSTM-based decoder has been approximately 9% higher than the best decoder from the same competition. However, there was no significant difference (Z = 1.540, p = 0.123). In order to verify the replication of the EEGNet-LSTM decoder on another data, we performed a test with PhysioNet's Physiobank EEG Motor Movement/Imagery dataset. The EEGNet-LSTM presented a higher performance (0.85 accuracy) than the EEGNet (0.82 accuracy). The results of this work can be important for the development of new research, as well as EEG-based BMI systems, which can benefit from the high precision of neural decoders.

A Deep Neural Network-Based Spike Sorting With Improved Channel Selection and Artefact Removal

In order to implement highly efficient brain-machine interface (BMI) systems, high-channel count sensing is often used to record extracellular action potentials. However, the extracellular recordings are typically severely contaminated by artefacts and various noise sources, rendering the separation of multi-unit neural recordings an immensely challenging task. Removing artefact and noise from neural events can improve the spike sorting performance and classification accuracy. This paper presents a deep learning technique called deep spike detection (DSD) with a strong learning ability of high-dimensional vectors for neural channel selection and artefacts removal from the selected neural channel. The proposed method significantly improves spike detection compared to the conventional methods by sequentially diminishing the noise level and discarding the active artefacts in the recording channels. The simulated and experimental results show that there is considerably better performance when the extracellular raw recordings are cleaned prior to assigning individual spikes to the neurons that generated them. The DSD achieves an overall classification accuracy of 91.53% and outperformes Wave_clus by 3.38% on the simulated dataset with various noise levels and artefacts.

The decoder design and performance comparative analysis for closed-loop brain–machine interface system

The decoder design and performance comparative analysis for closed-loop brain–machine interface system

Review of Closed-Loop Brain–Machine Interface Systems From a Control Perspective

In recent years, brain–machine interface (BMI) technology has made great progress in controlling external devices and restoring motor function for people with disabilities. To better optimize BMI system performance, in this article, we summarize and describe a universal closed-loop BMI system framework and review the latest developments over the past ten years from a control perspective. First, the basic BMI systems with open-loop and closed-loop structures are introduced in chronological order. Second, the units of the universal closed-loop BMI system, i.e., the decoder, encoder, and auxiliary controller, are reviewed and summarized in terms of principles, categories, and algorithms. Finally, from research and practical perspectives, the importance of biomimetic brain models, great challenges, and future developments are discussed based on current progress. With this analysis of the universal closed-loop framework, this review can provide necessary theoretical guidance for the research and development of BMI systems.

Neuroinflammatory Gene Expression Analysis Reveals Pathways of Interest as Potential Targets to Improve the Recording Performance of Intracortical Microelectrodes.

Intracortical microelectrodes are a critical component of brain-machine interface (BMI) systems. The recording performance of intracortical microelectrodes used for both basic neuroscience research and clinical applications of BMIs decreases over time, limiting the utility of the devices. The neuroinflammatory response to the microelectrode has been identified as a significant contributing factor to its performance. Traditionally, pathological assessment has been limited to a dozen or so known neuroinflammatory proteins, and only a few groups have begun to explore changes in gene expression following microelectrode implantation. Our initial characterization of gene expression profiles of the neuroinflammatory response to mice implanted with non-functional intracortical probes revealed many upregulated genes that could inform future therapeutic targets. Emphasis was placed on the most significant gene expression changes and genes involved in multiple innate immune sets, including Cd14, C3, Itgam, and Irak4. In previous studies, inhibition of Cluster of Differentiation 14 (Cd14) improved microelectrode performance for up to two weeks after electrode implantation, suggesting CD14 can be explored as a potential therapeutic target. However, all measures of improvements in signal quality and electrode performance lost statistical significance after two weeks. Therefore, the current study investigated the expression of genes in the neuroinflammatory pathway at the tissue-microelectrode interface in Cd14−/− mice to understand better how Cd14 inhibition was connected to temporary improvements in recording quality over the initial 2-weeks post-surgery, allowing for the identification of potential co-therapeutic targets that may work synergistically with or after CD14 inhibition to improve microelectrode performance.

Error-Related Potentials in Reinforcement Learning-Based Brain-Machine Interfaces.

The human brain has been an object of extensive investigation in different fields. While several studies have focused on understanding the neural correlates of error processing, advances in brain-machine interface systems using non-invasive techniques further enabled the use of the measured signals in different applications. The possibility of detecting these error-related potentials (ErrPs) under different experimental setups on a single-trial basis has further increased interest in their integration in closed-loop settings to improve system performance, for example, by performing error correction. Fewer works have, however, aimed at reducing future mistakes or learning. We present a review focused on the current literature using non-invasive systems that have combined the ErrPs information specifically in a reinforcement learning framework to go beyond error correction and have used these signals for learning.

A Power-Efficient Brain-Machine Interface System With a Sub-mw Feature Extraction and Decoding ASIC Demonstrated in Nonhuman Primates.

Intracortical brain-machine interfaces have shown promise for restoring function to people with paralysis, but their translation to portable and implantable devices is hindered by their high power consumption. Recent devices have drastically reduced power consumption compared to standard experimental brain-machine interfaces, but still require wired or wireless connections to computing hardware for feature extraction and inference. Here, we introduce a Neural Recording And Decoding (NeuRAD) application specific integrated circuit (ASIC) in 180 nm CMOS that can extract neural spiking features and predict two-dimensional behaviors in real-time. To reduce amplifier and feature extraction power consumption, the NeuRAD has a hardware accelerator for extracting spiking band power (SBP) from intracortical spiking signals and includes an M0 processor with a fixed-point Matrix Acceleration Unit (MAU) for efficient and flexible decoding. We validated device functionality by recording SBP from a nonhuman primate implanted with a Utah microelectrode array and predicting the one- and two-dimensional finger movements the monkey was attempting to execute in closed-loop using a steady-state Kalman filter (SSKF). Using the NeuRAD's real-time predictions, the monkey achieved 100% success rate and 0.82 s mean target acquisition time to control one-dimensional finger movements using just 581 μW. To predict two-dimensional finger movements, the NeuRAD consumed 588 μW to enable the monkey to achieve a 96% success rate and 2.4 s mean acquisition time. By employing SBP, ASIC brain-machine interfaces can close the gap to enable fully implantable therapies for people with paralysis.

Decoding neural activity preceding balance loss during standing with a lower-limb exoskeleton using an interpretable deep learning model

Objective: Falls are a leading cause of death in adults 65 and older. Recent efforts to restore lower-limb function in these populations have seen an increase in the use of wearable robotic systems; however, fall prevention measures in these systems require early detection of balance loss to be effective. Prior studies have investigated whether kinematic variables contain information about an impending fall, but few have examined the potential of using electroencephalography (EEG) as a fall-predicting signal and how the brain responds to avoid a fall. Approach: To address this, we decoded neural activity in a balance perturbation task while wearing an exoskeleton. We acquired EEG, electromyography (EMG), and center of pressure (COP) data from seven healthy participants during mechanical perturbations while standing. The timing of the perturbations was randomized in all trials. Main results: We found perturbation evoked potentials (PEP) components as early as 75–134 ms after the onset of the external perturbation, which preceded both the peak in EMG (∼180 ms) and the COP (∼350 ms). A convolutional neural network trained to predict balance perturbations from single-trial EEG had a mean F-score of 75.0 ± 4.3. Clustering GradCAM-based model explanations demonstrated that the model utilized components in the PEP and was not driven by artifacts. Additionally, dynamic functional connectivity results agreed with model explanations; the nodal connectivity measured using phase difference derivative was higher in the occipital-parietal region in the early stage of perturbations, before shifting to the parietal, motor, and back to the frontal-parietal channels. Continuous-time decoding of COP trajectories from EEG, using a gated recurrent unit model, achieved a mean Pearson’s correlation coefficient of 0.7 ± 0.06. Significance: Overall, our findings suggest that EEG signals contain short-latency neural information related to an impending fall, which may be useful for developing brain-machine interface systems for fall prevention in robotic exoskeletons.

Development of an Ergonomic User Interface Design of Calcium Imaging Processing System

An optical brain-machine interface (O-BMI) system using calcium imaging has various advantages such as high resolution, a comprehensive view of large neural populations, abilities such as long-term stable recording, and applicability to freely behaving animals in neuroscience research. The present study developed an ergonomic user interface (UI) design, based on a use scenario for an O-BMI system that can be used for the acquisition and processing of calcium imaging in freely behaving rodents. The UI design was developed in three steps: (1) identification of design and function requirements of users, (2) establishment of a use scenario, and (3) development of a UI prototype. The UI design requirements were identified by a literature review, a benchmark of existing systems, and a focus group interview with five neuroscience researchers. Then, the use scenario was developed for tasks of data acquisition, feature extraction, and neural decoding for offline and online processing by considering the sequences of operations and needs of users. Lastly, a digital prototype incorporating an information architecture, graphic user interfaces, and simulated functions was fabricated. A usability test was conducted with five neuroscientists (work experience = 3.4 ± 1.1 years) and five ergonomic experts (work experience = 3.6 ± 2.7 years) to compare the digital prototypes with four existing systems (Miniscope, nVista, Mosaic, and Suite2p). The usability testing results showed that the ergonomic UI design was significantly preferred to the UI designs of the existing systems by reducing the task completion time by 10.1% to 70.2% on average, the scan path length by 14.4% to 88.7%, and perceived workload by 12.2% to 37.9%, increasing satisfaction by 11.3% to 74.3% in data acquisition and signal-extraction tasks. The present study demonstrates the significance of the user-centered design approach in the development of a system for neuroscience research. Further research is needed to validate the usability test results of the UI prototype as a corresponding real system is implemented.

Algorithm and hardware considerations for real-time neural signal on-implant processing

Objective. Various on-workstation neural-spike-based brain machine interface (BMI) systems have reached the point of in-human trials, but on-node and on-implant BMI systems are still under exploration. Such systems are constrained by the area and battery. Researchers should consider the algorithm complexity, available resources, power budgets, CMOS technologies, and the choice of platforms when designing BMI systems. However, the effect of these factors is currently still unclear. Approaches. Here we have proposed a novel real-time 128 channel spike detection algorithm and optimised it on microcontroller (MCU) and field programmable gate array (FPGA) platforms towards consuming minimal power and memory/resources. It is presented as a use case to explore the different considerations in system design. Main results. The proposed spike detection algorithm achieved over 97% sensitivity and a smaller than 3% false detection rate. The MCU implementation occupies less than 3 KB RAM and consumes 31.5 µW ch−1. The FPGA platform only occupies 299 logic cells and 3 KB RAM for 128 channels and consumes 0.04 µW ch−1. Significance. On the spike detection algorithm front, we have eliminated the processing bottleneck by reducing the dynamic power consumption to lower than the hardware static power, without sacrificing detection performance. More importantly, we have explored the considerations in algorithm and hardware design with respect to scalability, portability, and costs. These findings can facilitate and guide the future development of real-time on-implant neural signal processing platforms.

Auxiliary controller design and performance comparative analysis in closed-loop brain-machine interface system.

Brain-machine interface (BMI) can realize information interaction between the brain and external devices, and yet the control accuracy is limited by the change of electroencephalogram signals. The introduction of auxiliary controller can overcome the above problems, but the performance of different auxiliary controllers is quite different. Hence, in this paper, we comprehensively compare and analyze the performance of different auxiliary controllers to provide a theoretical basis for designing BMI system. The main work includes: (1) designing four kinds of auxiliary controllers based on simultaneous perturbation stochastic approximation-function approximator (SPSA-FA), iterative feedback tuning-PID (IFT-PID), model predictive control (MPC) and model-free control (MFC); (2) based on the model of improved single-joint information transmission, constructing the closed-loop BMI systems with the decoder-based Wiener filter; and (3) comparing their performance in the constructed closed-loop BMI systems for dynamic motion restoration. The results show that the order of tracking accuracy is MPC, IFT-PID, SPSA-FA, MFC, and the order of time consumed is opposite. A good control effectiveness is achieved at the expense of time, so a suitable auxiliary controller should be selected according to the actual requirements.

Neuropathological effects of chronically implanted, intracortical microelectrodes in a tetraplegic patient

Objective. Intracortical microelectrode arrays (MEA) can be used as part of a brain–machine interface system to provide sensory feedback control of an artificial limb to assist persons with tetraplegia. Variability in functionality of electrodes has been reported but few studies in humans have examined the impact of chronic brain tissue responses revealed postmortem on electrode performance in vivo. Approach. In a tetraplegic man, recording MEAs were implanted into the anterior intraparietal area and Brodmann’s area 5 (BA5) of the posterior parietal cortex and a recording and stimulation array was implanted in BA1 of the primary somatosensory cortex (S1). The participant expired from unrelated causes seven months after MEA implantation. The underlying tissue of two of the three devices was processed for histology and electrophysiological recordings were assessed. Main results. Recordings of neuronal activity were obtained from all three MEAs despite meningeal encapsulation. However, the S1 array had a greater encapsulation, yielded lower signal quality than the other arrays and failed to elicit somatosensory percepts with electrical stimulation. Histological examination of tissues underlying S1 and BA5 implant sites revealed localized leptomeningeal proliferation and fibrosis, lymphocytic infiltrates, astrogliosis, and foreign body reaction around the electrodes. The BA5 recording site showed focal cerebral microhemorrhages and leptomeningeal vascular ectasia. The S1 site showed focal tissue damage including vascular recanalization, neuronal loss, and extensive subcortical white matter necrosis. The tissue response at the S1 site included hemorrhagic-induced injury suggesting a likely mechanism for reduced function of the S1 implant. Significance. Our findings are similar to those from animal studies with chronic intracortical implants and suggest that vascular disruption and microhemorrhage during device implantation are important contributors to overall array and individual electrode performance and should be a topic for future device development to mitigate tissue responses. Neurosurgical considerations are also discussed.