Intracortical Brain-computer Interface Research Articles

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface.

Creating an intracortical brain-computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue. We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to. We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network (RNN) decoder with 10-12 clusters. This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

Gesture encoding in human left precentral gyrus neuronal ensembles.

Understanding the cortical activity patterns driving dexterous upper limb motion has the potential to benefit a broad clinical population living with limited mobility through the development of novel brain-computer interface (BCI) technology. The present study examines the activity of ensembles of motor cortical neurons recorded using microelectrode arrays in the dominant hemisphere of two BrainGate clinical trial participants with cervical spinal cord injury as they attempted to perform a set of 48 different hand gestures. Although each participant displayed a unique organization of their respective neural latent spaces, it was possible to achieve classification accuracies of ~70% for all 48 gestures (and ~90% for sets of 10). Our results show that single unit ensemble activity recorded in a single hemisphere of human precentral gyrus has the potential to generate a wide range of gesture-related signals across both hands, providing an intuitive and diverse set of potential command signals for intracortical BCI use.

Less is more: selection from a small set of options improves BCI velocity control.

We designed the discrete direction selection (DDS) decoder for intracortical brain computer interface (iBCI) cursor control and showed that it outperformed currently used decoders in a human-operated real-time iBCI simulator and in monkey iBCI use. Unlike virtually all existing decoders that map between neural activity and continuous velocity commands, DDS uses neural activity to select among a small menu of preset cursor velocities. We compared closed-loop cursor control across four visits by each of 48 naïve, able-bodied human subjects using either DDS or one of three common continuous velocity decoders: direct regression with assist (an affine map from neural activity to cursor velocity), ReFIT, and the velocity Kalman Filter. DDS outperformed all three by a substantial margin. Subsequently, a monkey using an iBCI also had substantially better performance with DDS than with the Wiener filter decoder (direct regression decoder that includes time history). Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of simplifying online iBCI control.

Using an Intracortical Brain Computer Interface to Decode Contextual Modulation from Speech (P7-11.006)

Using an Intracortical Brain Computer Interface to Decode Contextual Modulation from Speech (P7-11.006)

An Event-Based Neural Compressive Telemetry With >11× Loss-Less Data Reduction for High-Bandwidth Intracortical Brain Computer Interfaces.

Intracortical brain-computer interfaces offer superior spatial and temporal resolutions, but face challenges as the increasing number of recording channels introduces high amounts of data to be transferred. This requires power-hungry data serialization and telemetry, leading to potential tissue damage risks. To address this challenge, this paper introduces an event-based neural compressive telemetry (NCT) consisting of 8 channel-rotating Δ-ADCs, an event-driven serializer supporting a proposed ternary address event representation protocol, and an event-based LVDS driver. Leveraging a high sparsity of extracellular spikes and high spatial correlation of the high-density recordings, the proposed NCT achieves a compression ratio of >11.4×, while consumes only 1 µW per channel, which is 127× more efficient than state of the art. The NCT well preserves the spike waveform fidelity, and has a low normalized RMS error <23% even with a spike amplitude down to only 31 µV.

Decoding the Spike-Band Subthreshold Motor Cortical Activity

Intracortical Brain-Computer Interfaces (iBCI) use single-unit activity (SUA), multiunit activity (MUA) and local field potentials (LFP) to control neuroprosthetic devices. SUA and MUA are usually extracted from the bandpassed recording through amplitude thresholding, while subthreshold data are ignored. Here, we show that subthreshold data can actually be decoded to determine behavioral variables with test set accuracy of up to 100%. Although the utility of SUA, MUA and LFP for decoding behavioral variables has been explored previously, this study investigates the utility of spike-band subthreshold activity exclusively. We provide evidence suggesting that this activity can be used to keep decoding performance at acceptable levels even when SUA quality is reduced over time. To the best of our knowledge, the signals that we derive from the subthreshold activity may be the weakest neural signals that have ever been extracted from extracellular neural recordings, while still being decodable with test set accuracy of up to 100%. These results are relevant for the development of fully data-driven and automated methods for amplitude thresholding spike-band extracellular neural recordings in iBCIs containing thousands of electrodes.

Multi-channel LFP recording data compression scheme using Cooperative PCA and Kalman Filter

The power consumption and wireless communication bandwidth restrict the channel count and amount of high-resolution intracortical Brain–Computer Interfaces(iBCIs). So, it is necessary to compress the captured data before transmission. Here, we propose a multi-channel data compression method on the correlation between channels for Local Field Potential(LFPs) recorded through Utah microelectrode arrays. The method improves the compression performance of the Stepwise Expectation-Maximization Principal Component Analysis(SEM-PCA) algorithm by implementing the Kalman filter. The multi-channel correlation is sampled with high resolution while the uniqueness of each channel is retained at a low rate. Then, the two parts of information are fused by the Kalman filter, and finally, the data is reconstructed with high precision and high rate. Our testing results verified the effectiveness of the proposed compression scheme. Compared to the traditional SEM-PCA, this algorithm significantly improved neural LFP reconstruction accuracy from 79.61% to 88%.

The influence of non-stationarity of spike signals on decoding performance in intracortical brain-computer interface: a simulation study.

Intracortical Brain-Computer Interfaces (iBCI) establish a new pathway to restore motor functions in individuals with paralysis by interfacing directly with the brain to translate movement intention into action. However, the development of iBCI applications is hindered by the non-stationarity of neural signals induced by the recording degradation and neuronal property variance. Many iBCI decoders were developed to overcome this non-stationarity, but its effect on decoding performance remains largely unknown, posing a critical challenge for the practical application of iBCI. To improve our understanding on the effect of non-stationarity, we conducted a 2D-cursor simulation study to examine the influence of various types of non-stationarities. Concentrating on spike signal changes in chronic intracortical recording, we used the following three metrics to simulate the non-stationarity: mean firing rate (MFR), number of isolated units (NIU), and neural preferred directions (PDs). MFR and NIU were decreased to simulate the recording degradation while PDs were changed to simulate the neuronal property variance. Performance evaluation based on simulation data was then conducted on three decoders and two different training schemes. Optimal Linear Estimation (OLE), Kalman Filter (KF), and Recurrent Neural Network (RNN) were implemented as decoders and trained using static and retrained schemes. In our evaluation, RNN decoder and retrained scheme showed consistent better performance under small recording degradation. However, the serious signal degradation would cause significant performance to drop eventually. On the other hand, RNN performs significantly better than the other two decoders in decoding simulated non-stationary spike signals, and the retrained scheme maintains the decoders' high performance when changes are limited to PDs. Our simulation work demonstrates the effects of neural signal non-stationarity on decoding performance and serves as a reference for selecting decoders and training schemes in chronic iBCI. Our result suggests that comparing to KF and OLE, RNN has better or equivalent performance using both training schemes. Performance of decoders under static scheme is influenced by recording degradation and neuronal property variation while decoders under retrained scheme are only influenced by the former one.

Decoding spatial locations from primate lateral prefrontal cortex neural activity during virtual navigation

Objective. Decoding the intended trajectories from brain signals using a brain-computer interface system could be used to improve the mobility of patients with disabilities. Approach. Neuronal activity associated with spatial locations was examined while macaques performed a navigation task within a virtual environment. Main results. Here, we provide proof of principle that multi-unit spiking activity recorded from the lateral prefrontal cortex (LPFC) of non-human primates can be used to predict the location of a subject in a virtual maze during a navigation task. The spatial positions within the maze that require a choice or are associated with relevant task events can be better predicted than the locations where no relevant events occur. Importantly, within a task epoch of a single trial, multiple locations along the maze can be independently identified using a support vector machine model. Significance. Considering that the LPFC of macaques and humans share similar properties, our results suggest that this area could be a valuable implant location for an intracortical brain-computer interface system used for spatial navigation in patients with disabilities.

Intracortical Hindlimb Brain–Computer Interface Systems: A Systematic Review

Brain-computer interfaces (BCI) can help people with motor disorders to regain their ability to communicate and interact with the surrounding environment. The majority of studies in this field pursue the development of BCI systems to enhance or restore the movement functionality of people with disability. Although the studies on the development of BCIs to restore hindlimb movements have shorter backgrounds compared to forelimb, several studies have investigated hindlimb BCIs and their results were promising. In the present study, we systematically reviewed the studies investigating the decoding of hindlimb movement parameters using intracortical signals. Three scientific databases (PubMed, Scopus, and Embase) were used to extract the articles and the experiment, recording, processing methods, and results of the included studies were discussed. Although several studies on upper-limb intracortical BCIs have been conducted on human subjects, almost all studies in hindlimb intracortical BCIs field were performed on animal subjects. The most investigated task was walking on a treadmill, and the position of hindlimb joints and gait phase were the most studied continuous and discrete parameters, respectively. The included studies have mainly used spikes and linear decoders, which leaves the question of the effectiveness of using local field potentials and nonlinear decoders in this field unanswered. Although the results imply that hindlimb movement decoding using brain signals is feasible in laboratory conditions, further investigations are required to examine the hindlimb BCIs in real-life conditions.

Neural Decoding for Intracortical Brain-Computer Interfaces.

Brain-computer interfaces have revolutionized the field of neuroscience by providing a solution for paralyzed patients to control external devices and improve the quality of daily life. To accurately and stably control effectors, it is important for decoders to recognize an individual's motor intention from neural activity either by noninvasive or intracortical neural recording. Intracortical recording is an invasive way of measuring neural electrical activity with high temporal and spatial resolution. Herein, we review recent developments in neural signal decoding methods for intracortical brain-computer interfaces. These methods have achieved good performance in analyzing neural activity and controlling robots and prostheses in nonhuman primates and humans. For more complex paradigms in motor rehabilitation or other clinical applications, there remains more space for further improvements of decoders.

Heterogeneous domain adaptation for intracortical signal classification using domain consensus

Most domain adaptation studies have made efforts to solve the problem of homogeneous data shifts, which share the same feature and label space in both domains. However, for the heterogeneous scenarios, it still needs improvements in practical applications for intracortical brain-computer interface (iBCI) systems. To address this issue, we propose a domain adaptation framework for decoding reaching and grasping movements of iBCI systems in two heterogeneous scenarios, i.e., partial domain adaptation (PDA) and open set domain adaptation (OSDA). The proposed framework combined the neural activity vector (NAV) features and the domain consensus clustering (DCC) method for iBCI classification. For heterogeneous scenarios, the framework can map common classes according to the clusters with the highest domain consensus score at the semantic-level and distinguish private classes at the sample-level. We evaluated the framework on the data, which were collected from three array electrodes implanted in three regions of cerebral cortexes from one monkey across two sessions, each consisting of four continuous days. The proposed framework outperformed than four PDA and two OSDA methods, which effectively reduced the disparity of feature and label space in two heterogeneous scenarios. Consequently, our results demonstrated that the proposed framework is a new solution for decoder calibration in two heterogeneous scenarios, and it facilitates the generality of the iBCI system for clinical applications.

Decoder calibration framework for intracortical brain-computer interface system via domain adaptation

Intracortical brain-computer interface (iBCI) systems have evolved into a clinical approach to assist people with disabilities and paralysis. The decoder is one of the most crucial parts of iBCI systems. In reality, it is difficult to maintain efficient performance with a fixed decoder due to frequent changes in the distribution of data recorded over time. To improve the reliability of iBCI systems, usually, it is necessary to recalibrate decoders every time with a large amount of current data. However, it is difficult to obtain enough current data for recalibration. To reduce the recalibration data, we propose a domain adaptation-based decoder calibration framework (DA-DCF) for decoding reach-and grasp movements, which utilizes a small amount of current data and robustly achieves good performance. In the DA-DCF, a source convolutional neural network (CNN) learned representation from source data to initialize the target CNN. Secondly, the target CNN was trained through a confusing domain discriminator until it would not distinguish between the source and target domains. Thirdly, the features transformed by the target CNN were classified by the classifier for movements in the target domain. Finally, the practicality and efficiency of the DA-DCF were evaluated with other comparative frameworks. The comparative experiments were conducted on the data from a male rhesus monkey, which was trained to perform three categories of reach-and-grasp tasks across two sessions. Our results showed that DA-DCF diminished the disparity between source and target data, and significantly reduced the amount of recalibration data while improving the decoding accuracy. It is plausible to utilize DA-DCF in real-time iBCI systems of the forelimb movement control.

BCImat: a Matlab-based framework for Intracortical Brain-Computer Interfaces and their simulation with an artificial spiking neural network

Ferrea et al., (2022). BCImat: a Matlab-based framework for Intracortical Brain-Computer Interfaces and their simulation with an artificial spiking neural network. Journal of Open Source Software, 7(75), 3956, https://doi.org/10.21105/joss.03956

A 1.66Gb/s and 5.8pJ/b Transcutaneous IR-UWB Telemetry System with Hybrid Impulse Modulation for Intracortical Brain-Computer Interfaces.

Intra-cortical extracellular neural sensing is being rapidly and widely applied in several clinical research and brain-computer interfaces (BCIs), as the number of sensing channels continues to double every 6 years. By distributing multiple high-density extracellular micro-electrode arrays (MEAs) in vivo across the brain, each with 1000's of sensing channels, neuroscientists have begun to map the correlation of neuronal activity across different brain regions, with single-neuron precision [1]. Since each neural sensing channel typically samples at 20 to 50kS/s with a > 10b ADC, multiple MEAs demand a data transfer rate up to Gb/s [2]. However, these BCIs are severely hindered in many clinical uses due to the lack of a high-data-rate and miniature-wireless-telemetry solution that can be implanted below the scalp, i.e., transcutaneously (Fig. 24.2.1). The area of the wireless telemetry module should be miniaturized to ~3cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> due to neurosurgical implantation constraints. A transmission range up to 10cm is highly desirable, in order to improve the reliability of the wireless link against e.g., antenna misalignment, etc. Finally, the power consumption of the wireless telemetry should be limited to ~10mW to minimize thermal flux from the module's surface area, avoiding excessive tissue heating. Most of the conventional transcutaneous wireless telemetry systems adopt inductive coupling, but the data-rate is limited to a few Mb/s. A near-infrared (NIR) optical transcutaneous TX using a vertical-cavity-surface-emitting laser (VCSEL) [2] demonstrated a data-rate up to 300Mb/s but suffers from a limited transmission range (4mm) and requires a sub-mm precise alignment between the implant TX and a wearable RX. Impulse-radio UWB (IR-UWB) is promising for the targeted requirements [3]–[5].

The impact of distractions on intracortical brain–computer interface control of a robotic arm

ABSTRACT The objective of the study was to evaluate the impact of distractions on intracortical brain–computer interface (BCI) performance. Two individuals with tetraplegia had microelectrode arrays implanted into their motor cortex. Each used a BCI to move a robotic arm between two targets as quickly as possible, performed alone and with distractions. Performance outcomes were similar across conditions, except for a drop in the median path efficiency in the motor action distraction for Subject P2 (p = 0.033, Mann–Whitney U-test). Other distraction conditions resulted in intermittent impairments in performance, but were not statistically different from the no distraction condition. Both subjects rated the overall difficulty of the task with and without distractions as low. Consistent levels of BCI performance were mirrored by small differences in neural firing rates across conditions, demonstrating that experienced BCI users can maintain performance on a simple reaching task under these various conditions.

Selection of Essential Neural Activity Timesteps for Intracortical Brain-Computer Interface Based on Recurrent Neural Network.

Intracortical brain–computer interfaces (iBCIs) translate neural activity into control commands, thereby allowing paralyzed persons to control devices via their brain signals. Recurrent neural networks (RNNs) are widely used as neural decoders because they can learn neural response dynamics from continuous neural activity. Nevertheless, excessively long or short input neural activity for an RNN may decrease its decoding performance. Based on the temporal attention module exploiting relations in features over time, we propose a temporal attention-aware timestep selection (TTS) method that improves the interpretability of the salience of each timestep in an input neural activity. Furthermore, TTS determines the appropriate input neural activity length for accurate neural decoding. Experimental results show that the proposed TTS efficiently selects 28 essential timesteps for RNN-based neural decoders, outperforming state-of-the-art neural decoders on two nonhuman primate datasets ( for monkey Indy and for monkey N). In addition, it reduces the computation time for offline training (reducing 5–12%) and online prediction (reducing 16–18%). When visualizing the attention mechanism in TTS, the preparatory neural activity is consecutively highlighted during arm movement, and the most recent neural activity is highlighted during the resting state in nonhuman primates. Selecting only a few essential timesteps for an RNN-based neural decoder provides sufficient decoding performance and requires only a short computation time.

Home Use of a Percutaneous Wireless Intracortical Brain-Computer Interface by Individuals With Tetraplegia.

Objective.Individuals with neurological disease or injury such as amyotrophic lateral sclerosis, spinal cord injury or stroke may become tetraplegic, unable to speak or even locked-in. For people with these conditions, current assistive technologies are often ineffective. Brain-computer interfaces are being developed to enhance independence and restore communication in the absence of physical movement. Over the past decade, individuals with tetraplegia have achieved rapid on-screen typing and point-and-click control of tablet apps using intracortical brain-computer interfaces (iBCIs) that decode intended arm and hand movements from neural signals recorded by implanted microelectrode arrays. However, cables used to convey neural signals from the brain tether participants to amplifiers and decoding computers and require expert oversight, severely limiting when and where iBCIs could be available for use. Here, we demonstrate the first human use of a wireless broadband iBCI.Methods.Based on a prototype system previously used in pre-clinical research, we replaced the external cables of a 192-electrode iBCI with wireless transmitters and achieved high-resolution recording and decoding of broadband field potentials and spiking activity from people with paralysis. Two participants in an ongoing pilot clinical trial completed on-screen item selection tasks to assess iBCI-enabled cursor control.Results:Communication bitrates were equivalent between cabled and wireless configurations. Participants also used the wireless iBCI to control a standard commercial tablet computer to browse the web and use several mobile applications. Within-day comparison of cabled and wireless interfaces evaluated bit error rate, packet loss, and the recovery of spike rates and spike waveforms from the recorded neural signals. In a representative use case, the wireless system recorded intracortical signals from two arrays in one participant continuously through a 24-hour period at home.Significance.Wireless multi-electrode recording of broadband neural signals over extended periods introduces a valuable tool for human neuroscience research and is an important step toward practical deployment of iBCI technology for independent use by individuals with paralysis. On-demand access to high-performance iBCI technology in the home promises to enhance independence and restore communication and mobility for individuals with severe motor impairment.

High-performance brain-to-text communication via handwriting

Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. So far, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping1-5 or point-and-click typing with a computer cursor6,7. However, rapid sequences of highly dexterous behaviours, such as handwriting or touch typing, might enable faster rates of communication. Here we developed an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time, using a recurrent neural network decoding approach. With this BCI, our study participant, whose hand was paralysed from spinal cord injury, achieved typing speeds of 90characters per minute with 94.1% raw accuracy online, and greater than 99% accuracy offline with a general-purpose autocorrect. To our knowledge, these typing speeds exceed those reported for any other BCI, and are comparable to typical smartphone typing speeds of individuals in the age group of our participant (115characters per minute)8. Finally, theoretical considerations explain why temporally complex movements, such as handwriting, may be fundamentally easier to decode than point-to-point movements. Our results open a new approach for BCIs and demonstrate the feasibility of accurately decoding rapid, dexterous movements years after paralysis.

Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex.

Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.