Intracortical Brain-machine Interfaces Research Articles

A new approach for neural decoding by inspiring of hyperdimensional computing for implantable intra-cortical BMIs

In the field of Brain Machine Interface (BMI), the process of translating motor intention into a machine command is denoted as decoding. However, despite recent advancements, decoding remains a formidable challenge within BMI. The utilization of current decoding algorithms in the field of BMI often involves computational complexity and requires the use of computers. This is primarily due to the reliance on mathematical models to address the decoding issue and perform subsequent output calculations. Unfortunately, computers are not feasible for implantable BMI systems due to their size and power consumption. To address this predicament, this study proposes a pioneering approach inspired by hyperdimensional computing. This approach first involves identifying the pattern of each stimulus by considering the normal firing rate distribution of each neuron. Subsequently, the newly observed firing pattern for each input is compared with the patterns detected at each moment for each neuron. The algorithm, which shares similarities with hyperdimensional computing, identifies the most similar pattern as the final output. This approach reduces the dependence on mathematical models. The efficacy of this method is assessed through the utilization of an authentic dataset acquired from the Frontal Eye Field (FEF) of two male rhesus monkeys. The output space encompasses eight possible angles. The results demonstrate an accuracy rate of 51.5% while exhibiting significantly low computational complexity, involving a mere 2050 adder operators. Furthermore, the proposed algorithm is implemented on a field-programmable gate array (FPGA) and as an ASIC designe in a standard CMOS 180 nm technology, underscoring its suitability for real-time implantable BMI applications. The implementation required only 2.3 Kbytes of RAM, occupied an area of 2.2 mm2, and consumed 9.32 µW at a 1.8 V power supply. Consequently, the proposed solution represents an accurate, low computational complexity, hardware-friendly, and real-time approach.

Functional Electrical Stimulation and Brain-Machine Interfaces for Simultaneous Control of Wrist and Finger Flexion.

Brain-machine interface (BMI) controlled functional electrical stimulation (FES) is a promising treatment to restore hand movements to people with cervical spinal cord injury. Recent intracortical BMIs have shown unprecedented successes in decoding user intentions, however the hand movements restored by FES have largely been limited to predetermined grasps. Restoring dexterous hand movements will require continuous control of many biomechanically linked degrees-of-freedom in the hand, such as wrist and finger flexion, that would form the basis of those movements. Here we investigate the ability to restore simultaneous wrist and finger flexion, which would enable grasping with a controlled hand posture and assist in manipulating objects once grasped. We demonstrate that intramuscular FES can enable monkeys with temporarily paralyzed hands to move their fingers and wrist across a functional range of motion, spanning an average 88.6 degrees at the metacarpophalangeal joint flexion and 71.3 degrees of wrist flexion, and intramuscular FES can control both joints simultaneously in a real-time task. Additionally, we demonstrate a monkey using an intracortical BMI to control the wrist and finger flexion in a virtual hand, both before and after the hand is temporarily paralyzed, even achieving success rates and acquisition times equivalent to able-bodied control with BMI control after temporary paralysis in two sessions. Together, this outlines a method using an artificial brain-to-body interface that could restore continuous wrist and finger movements after spinal cord injury.

Calibration-Free and Hardware-Efficient Neural Spike Detection for Brain Machine Interfaces.

Recent translational efforts in brain-machine interfaces (BMI) are demonstrating the potential to help people with neurological disorders. The current trend in BMI technology is to increase the number of recording channels to the thousands, resulting in the generation of vast amounts of raw data. This in turn places high bandwidth requirements for data transmission, which increases power consumption and thermal dissipation of implanted systems. On-implant compression and/or feature extraction are therefore becoming essential to limiting this increase in bandwidth, but add further power constraints - the power required for data reduction must remain less than the power saved through bandwidth reduction. Spike detection is a common feature extraction technique used for intracortical BMIs. In this article, we develop a novel firing-rate-based spike detection algorithm that requires no external training and is hardware efficient and therefore ideally suited for real-time applications. Key performance and implementation metrics such as detection accuracy, adaptability in chronic deployment, power consumption, area utilization, and channel scalability are benchmarked against existing methods using various datasets. The algorithm is first validated using a reconfigurable hardware (FPGA) platform and then ported to a digital ASIC implementation in both 65 nm and 0.18 μm CMOS technologies. The 128-channel ASIC design implemented in a 65 nm CMOS technology occupies 0.096 mm2 silicon area and consumes 4.86 μW from a 1.2 V power supply. The adaptive algorithm achieves a 96% spike detection accuracy on a commonly used synthetic dataset, without the need for any prior training.

Intracortical Brain-machine Interface for Restoring Sensory Motor Function: Progress and Challenges

Limb loss or paralysis due to spinal cord injury has a devastating impact on quality of life. One way to restore the sensory and motor abilities lost by amputees and quadriplegics is to provide them with implants that interface directly with the central nervous system. Such Brain-machine interfaces could enable patients to exert active control over the electrical contractions of prosthetic limbs or paralysed muscles. The parallel interface can transmit sensory information about these motor outcomes back to the patient. Recent developments in algorithms for decoding motor intention from neuronal activity, using biomimetic and adaptation-based approaches and methods for delivering sensory feedback through electrical stimulation of neurons have shown promise for invasive interfaces with sensorimotor cortex, although significant challenges remain.

Hardware-Efficient Compression of Neural Multi-Unit Activity

Brain-machine interfaces (BMI) are tools for treating neurological disorders and motor-impairments. It is essential that the next generation of intracortical BMIs is wireless so as to remove percutaneous connections, i.e. wires, and the associated mechanical and infection risks. This is required for the effective translation of BMIs into clinical applications and is one of the remaining bottlenecks. However, due to cortical tissue thermal dissipation safety limits, the on-implant power consumption must be strictly limited. Therefore, both the neural signal processing and wireless communication power should be minimal, while the implants should provide signals that offer high behavioural decoding performance (BDP). The Multi-Unit Activity (MUA) signal is the most common signal in modern BMIs. However, with an ever-increasing channel count, the raw data bandwidth is becoming prohibitively high due to the associated communication power exceeding the safety limits. Data compression is therefore required. To meet this need, this work developed hardware-efficient static Huffman compression schemes for MUA data. Our final system reduced the bandwidth to 27 bps/channel, compared to the standard MUA rate of 1 kbps/channel. This compression is over an order of magnitude more than has been achieved before, while using only 0.96 uW/channel processing power and 246 logic cells. Our results were verified on 3 datasets and less than 1% loss in BDP was observed. As such, with the use of effective data compression, an order more of MUA channels can be fitted on-implant, enabling the next generation of high-performance wireless intracortical BMIs.

Real-time linear prediction of simultaneous and independent movements of two finger groups using an intracortical brain-machine interface

Modern brain-machine interfaces can return function to people with paralysis, but current upper extremity brain-machine interfaces are unable to reproduce control of individuated finger movements. Here, for the first time, we present a real-time, high-speed, linear brain-machine interface in nonhuman primates that utilizes intracortical neural signals to bridge this gap. We created a non-prehensile task that systematically individuates two finger groups, the index finger and the middle-ring-small fingers combined. During online brain control, the ReFIT Kalman filter could predict individuated finger group movements with high performance. Next, training ridge regression decoders with individual movements was sufficient to predict untrained combined movements and vice versa. Finally, we compared the postural and movement tuning of finger-related cortical activity to find that individual cortical units simultaneously encode multiple behavioral dimensions. Our results suggest that linear decoders may be sufficient for brain-machine interfaces to execute high-dimensional tasks with the performance levels required for naturalistic neural prostheses.

Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning

Objective. Brain–machine interfaces (BMIs) seek to restore lost motor functions in individuals with neurological disorders by enabling them to control external devices directly with their thoughts. This work aims to improve robustness and decoding accuracy that currently become major challenges in the clinical translation of intracortical BMIs. Approach. We propose entire spiking activity (ESA)—an envelope of spiking activity that can be extracted by a simple, threshold-less, and automated technique—as the input signal. We couple ESA with deep learning-based decoding algorithm that uses quasi-recurrent neural network (QRNN) architecture. We evaluate comprehensively the performance of ESA-driven QRNN decoder for decoding hand kinematics from neural signals chronically recorded from the primary motor cortex area of three non-human primates performing different tasks. Main results. Our proposed method yields consistently higher decoding performance than any other combinations of the input signal and decoding algorithm previously reported across long-term recording sessions. It can sustain high decoding performance even when removing spikes from the raw signals, when using the different number of channels, and when using a smaller amount of training data. Significance. Overall results demonstrate exceptionally high decoding accuracy and chronic robustness, which is highly desirable given it is an unresolved challenge in BMIs.

Feature-Selection-Based Transfer Learning for Intracortical Brain–Machine Interface Decoding

The time spent in collecting current samples for decoder calibration and the computational burden brought by high-dimensional neural recordings remain two challenging problems in intracortical brain-machine interfaces (iBMIs). Decoder calibration optimization approaches have been proposed, and neuron selection methods have been used to reduce computational burden. However, few methods can solve both problems simultaneously. In this article, we present a symmetrical-uncertainty-based transfer learning (SUTL) method that combines transfer learning with feature selection. The proposed method uses symmetrical uncertainty to quantitatively measure three indices for feature selection: stationarity, importance and redundancy of the feature. By selecting the stationary features, the disparities between the historical data and current data can be diminished, and the historical data can be effectively used for decoder calibration, thereby reducing the demand for current data. After selecting the important and non-redundant features, only the channels corresponding to them need to work; thus, the computational burden is reduced. The proposed method was tested on neural data recorded from two rhesus macaques to decode the reaching position or grasping gesture. The results showed that the SUTL method diminished the disparities between the historical data and current data, while achieving superior decoding performance with the needs of only ten current samples each category, less than 10% the number of features and 30% the number of neural recording channels. Additionally, unlike most studies on iBMIs, feature selection was implemented instead of neuron selection, and the average decoding accuracy achieved by the former was 6.6% higher.

Finding Kinematics-Driven Latent Neural States From Neuronal Population Activity for Motor Decoding.

While intracortical brain-machine interfaces (BMIs) demonstrate feasibility to restore mobility to people with paralysis, it is still challenging to maintain high-performance decoding in clinical BMIs. One of the main obstacles for high-performance BMI is the noise-prone nature of traditional decoding methods that connect neural response explicitly with physical quantity, such as velocity. In contrast, the recent development of latent neural state model enables a robust readout of large-scale neuronal population activity contents. However, these latent neural states do not necessarily contain kinematic information useful for decoding. Therefore, this study proposes a new approach to finding kinematics-dependent latent factors by extracting latent factors' kinematics-dependent components using linear regression. We estimated these components from the population activity through nonlinear mapping. The proposed kinematics-dependent latent factors generate neural trajectories that discriminate latent neural states before and after the motion onset. We compared the decoding performance of the proposed analysis model with the results from other popular models. They are factor analysis (FA), Gaussian process factor analysis (GPFA), latent factor analysis via dynamical systems (LFADS), preferential subspace identification (PSID), and neuronal population firing rates. The proposed analysis model results in higher decoding accuracy than do the others ( % improvement on average). Our approach may pave a new way to extract latent neural states specific to kinematic information from motor cortices, potentially improving decoding performance for online intracortical BMIs.

Multi-source domain adaptation for decoder calibration of intracortical brain-machine interface

Objective. For nonstationarity of neural recordings, daily retraining is required in the decoder calibration of intracortical brain-machine interfaces (iBMIs). Domain adaptation (DA) has started to be applied in iBMIs to solve the problem of daily retraining by taking advantage of historical data. However, previous DA studies used only a single source domain, which might lead to performance instability. In this study, we proposed a multi-source DA algorithm, by fully utilizing the historical data, to achieve a better and more robust decoding performance while reducing the decoder calibration time. Approach. The neural signals were recorded from two rhesus macaques using intracortical electrodes to decode the reaching and grasping movements. A principal component analysis (PCA)-based multi-source domain adaptation (PMDA) algorithm was proposed to apply the feature transfer to diminish the disparities between the target domain and each source domain. Moreover, the multiple weighted sub-classifiers based on multi-source domain data and small current sample set were constructed to accomplish the decoding. Main results. Our algorithm was able to make use of the multi-source domain data and achieve better and more robust decoding performance compared with other methods. Only a small current sample set was needed by our algorithm in order for the decoder calibration time to be effectively reduced. Significance. (1) The idea of the multi-source DA was introduced into the iBMIs to solve the problem of time consumption in the daily decoder retraining. (2) Instead of using only single-source domain data in the previous study, our algorithm made use of multi-day historical data, resulting in better and more robust decoding performance. (3) Our algorithm could be accomplished with only a small current sample set, and it can effectively reduce the decoder calibration time, which is important for further clinical applications.

Decoding Kinematic Information From Primary Motor Cortex Ensemble Activities Using a Deep Canonical Correlation Analysis.

The control of arm movements through intracortical brain–machine interfaces (BMIs) mainly relies on the activities of the primary motor cortex (M1) neurons and mathematical models that decode their activities. Recent research on decoding process attempts to not only improve the performance but also simultaneously understand neural and behavioral relationships. In this study, we propose an efficient decoding algorithm using a deep canonical correlation analysis (DCCA), which maximizes correlations between canonical variables with the non-linear approximation of mappings from neuronal to canonical variables via deep learning. We investigate the effectiveness of using DCCA for finding a relationship between M1 activities and kinematic information when non-human primates performed a reaching task with one arm. Then, we examine whether using neural activity representations from DCCA improves the decoding performance through linear and non-linear decoders: a linear Kalman filter (LKF) and a long short-term memory in recurrent neural networks (LSTM-RNN). We found that neural representations of M1 activities estimated by DCCA resulted in more accurate decoding of velocity than those estimated by linear canonical correlation analysis, principal component analysis, factor analysis, and linear dynamical system. Decoding with DCCA yielded better performance than decoding the original FRs using LSTM-RNN (6.6 and 16.0% improvement on average for each velocity and position, respectively; Wilcoxon rank sum test, p < 0.05). Thus, DCCA can identify the kinematics-related canonical variables of M1 activities, thus improving the decoding performance. Our results may help advance the design of decoding models for intracortical BMIs.

Classifying Intracortical Brain-Machine Interface Signal Disruptions Based on System Performance and Applicable Compensatory Strategies: A Review.

Brain-machine interfaces (BMIs) record and translate neural activity into a control signal for assistive or other devices. Intracortical microelectrode arrays (MEAs) enable high degree-of-freedom BMI control for complex tasks by providing fine-resolution neural recording. However, chronically implanted MEAs are subject to a dynamic in vivo environment where transient or systematic disruptions can interfere with neural recording and degrade BMI performance. Typically, neural implant failure modes have been categorized as biological, material, or mechanical. While this categorization provides insight into a disruption's causal etiology, it is less helpful for understanding degree of impact on BMI function or possible strategies for compensation. Therefore, we propose a complementary classification framework for intracortical recording disruptions that is based on duration of impact on BMI performance and requirement for and responsiveness to interventions: (1) Transient disruptions interfere with recordings on the time scale of minutes to hours and can resolve spontaneously; (2) Reversible disruptions cause persistent interference in recordings but the root cause can be remedied by an appropriate intervention; (3) Irreversible compensable disruptions cause persistent or progressive decline in signal quality, but their effects on BMI performance can be mitigated algorithmically; and (4) Irreversible non-compensable disruptions cause permanent signal loss that is not amenable to remediation or compensation. This conceptualization of intracortical BMI disruption types is useful for highlighting specific areas for potential hardware improvements and also identifying opportunities for algorithmic interventions. We review recording disruptions that have been reported for MEAs and demonstrate how biological, material, and mechanical mechanisms of disruption can be further categorized according to their impact on signal characteristics. Then we discuss potential compensatory protocols for each of the proposed disruption classes. Specifically, transient disruptions may be minimized by using robust neural decoder features, data augmentation methods, adaptive machine learning models, and specialized signal referencing techniques. Statistical Process Control methods can identify reparable disruptions for rapid intervention. In-vivo diagnostics such as impedance spectroscopy can inform neural feature selection and decoding models to compensate for irreversible disruptions. Additional compensatory strategies for irreversible disruptions include information salvage techniques, data augmentation during decoder training, and adaptive decoding methods to down-weight damaged channels.

Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain-Machine Interface.

Background: For the nonstationarity of neural recordings in intracortical brain–machine interfaces, daily retraining in a supervised manner is always required to maintain the performance of the decoder. This problem can be improved by using a reinforcement learning (RL) based self-recalibrating decoder. However, quickly exploring new knowledge while maintaining a good performance remains a challenge in RL-based decoders. Methods: To solve this problem, we proposed an attention-gated RL-based algorithm combining transfer learning, mini-batch, and weight updating schemes to accelerate the weight updating and avoid over-fitting. The proposed algorithm was tested on intracortical neural data recorded from two monkeys to decode their reaching positions and grasping gestures. Results: The decoding results showed that our proposed algorithm achieved an approximate 20% increase in classification accuracy compared to that obtained by the non-retrained classifier and even achieved better classification accuracy than the daily retraining classifier. Moreover, compared with a conventional RL method, our algorithm improved the accuracy by approximately 10% and the online weight updating speed by approximately 70 times. Conclusions: This paper proposed a self-recalibrating decoder which achieved a good and robust decoding performance with fast weight updating and might facilitate its application in wearable device and clinical practice.

Lossless Compression of Intracortical Extracellular Neural Recordings using Non-Adaptive Huffman Encoding.

This paper investigates the effectiveness of four Huffman-based compression schemes for different intracortical neural signals and sample resolutions. The motivation is to find effective lossless, low-complexity data compression schemes for Wireless Intracortical Brain-Machine Interfaces (WI-BMI). The considered schemes include pre-trained Lone 1st and 2nd order encoding [1], pre-trained Delta encoding, and pre-trained Linear Neural Network Time (LNNT) encoding [2]. Maximum codeword-length limited versions are also considered to protect against overfit to training data. The considered signals are the Extracellular Action Potential signal, the Entire Spiking Activity signal, and the Local Field Potential signal. Sample resolutions of 5 to 13 bits are considered. The result show that overfit-protection dramatically improves compression, especially at higher sample resolutions. Across signals, 2nd order encoding generally performed best at lower sample resolutions, and 1st order, Delta and LNNT encoding performed best at higher sample resolutions. The proposed methods should generalise to other remote sensing applications where the distribution of the sensed data can be estimated a priori.

Sparse Ensemble Machine Learning to Improve Robustness of Long-Term Decoding in iBMIs.

This paper presents a novel sparse ensemble based machine learning approach to enhance robustness of intracortical Brain Machine Interfaces (iBMIs) in the face of non-stationary distribution of input neural data across time. Each classifier in the ensemble is trained on a randomly sampled (with replacement) set of input channels. These sparse connections ensure that with a high chance, few of the base classifiers should be less affected by the variations in some of the recording channels. We have tested the generality of this technique on different base classifiers - linear discriminant analysis (LDA), support vector machine (SVM), extreme learning machine (ELM) and multilayer perceptron (MLP). Results show decoding accuracy improvements of up to ≈21 %, 13%, 19%, 10% in non-human primate (NHP) A and 7%, 9%, 7%, 9% in NHP B across test days while using the sparse ensemble approach over a single classifier model for LDA, SVM, ELM and MLP algorithms respectively. Furthermore, improvements of up to ≈7(14)%, 8(15)%, 9(19)%, 7(15)% in NHP A and 8(15)%, 12(20)%, 15(23)%, 12(19)% in NHP B over Random Forest (Long-short Term Memory) have been obtained by sparse ensemble LDA, SVM, ELM, MLP respectively.

Towards Intelligent Intracortical BMI (i 2BMI): Low-Power Neuromorphic Decoders That Outperform Kalman Filters.

Fully-implantable wireless intracortical Brain Machine Interfaces (iBMI) is one of the most promising next frontiers in the nascent field of neurotechnology. However, scaling the number of channels in such systems by another 10× is difficult due to power and bandwidth requirements of the wireless transmitter. One promising solution for that is to include more processing, up to the decoder, in the implant so that transmission data-rate is reduced drastically. Earlier work on neuromorphic decoder chips only showed classification of discrete states. We present results for continuous state decoding using a low-power neuromorphic decoder chip termed Spike-input Extreme Learning Machine (SELMA) that implements a nonlinear decoder without memory and its memory-based version with time-delayed bins, SELMA-bins. We have compared SELMA, SELMA-bins against state-of-the-art Steady-State Kalman Filter (SSKF), a linear decoder with memory, across two different datasets involving a total of 4 non-human primates (NHPs). Results show at least a 10% (20%) or more increase in the fraction of variance accounted for (FVAF) by SELMA (SELMA-bins) over SSKF across the datasets. Estimated energy consumption comparison shows SELMA (SELMA-bins) consuming ≈ 9 nJ/update (23 nJ/update) against SSKF's ≈ 7.4 nJ/update for an iBMI with a 10 degree of freedom control. Thus, SELMA yields better performance against SSKF while consuming energy in the same range as SSKF whereas SELMA-bins performs the best with moderately increased energy consumption, albeit far less than energy required for raw data transmission. This paves the way for reducing transmission data rates in future scaled iBMI systems.

Reliability of motor and sensory neural decoding by threshold crossings for intracortical brain–machine interface

Objective. For intracortical neurophysiological studies, spike sorting is an important procedure to isolate single units for analyzing specific functions. However, whether spike sorting is necessary or not for neural decoding applications is controversial. Several studies showed that using threshold crossings (TC) instead of spike sorting could also achieve a similar satisfactory performance. However, such studies were limited in similar behavioral tasks, and the neural signal source mainly focused on the motor-related cortical regions. It is not certain if this conclusion is applicable to other situations. Therefore, we compared the performance of TC and spike sorting in neural decoding with more comprehensive paradigms and parameters. Approach. Two rhesus macaques implanted with Utah or floating microelectrode arrays (FMAs) in motor or sensory-related cortical regions were trained to perform a motor or a sensory task. Data from each monkey were preprocessed with three different schemes: TC, automatic sorting (AS), and manual sorting (MS). A support vector machine was used as the decoder, and the decoding accuracy was used for evaluating the performance of three preprocessing methods. Different neural signal sources, different decoders, and related parameters and decoding stability were further tested to systematically compare three preprocessing methods. Main results. TC could achieve a similar (−4.5 RMS threshold) or better (−3.0 RMS threshold) decoding performance compared to the other two sorting methods in the motor or sensory tasks even if the neural signal sources or decoder-related parameters were changed. Moreover, TC was much more stable in neural decoding across sessions and robust to changes of threshold. Significance. Our results indicated that spike-firing patterns could be stably extracted through TC from multiple cortices in both motor and sensory neural decoding applications. Considering the stability of TC, it might be more suitable for neural decoding compared to sorting methods.

Intrinsic Variable Learning for Brain-Machine Interface Control by Human Anterior Intraparietal Cortex

Although animal studies provided significant insights in understanding the neural basis of learning and adaptation, they often cannot dissociate between different learning mechanisms due to the lack of verbal communication. To overcome this limitation, we examined the mechanisms of learning and its limits in a human intracortical brain-machine interface (BMI) paradigm. A tetraplegic participant controlled a 2D computer cursor by modulating single-neuron activity in the anterior intraparietal area (AIP). By perturbing the neuron-to-movement mapping, the participant learned to modulate the activity of the recorded neurons to solve the perturbations by adopting a target re-aiming strategy. However, when no cognitive strategies were adequate to produce correct responses, AIP failed to adapt to perturbations. These findings suggest that learning is constrained by the pre-existing neuronal structure, although it is possible that AIP needs more training time to learn to generate novel activity patterns when cognitive re-adaptation fails to solve the perturbations.

Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements

Brain-machine interfaces (BMIs) that decode movement intentions should ignore neural modulation sources distinct from the intended command. However, neurophysiology and control theory suggest that motor cortex reflects the motor effector’s position, which could be a nuisance variable. We investigated motor cortical correlates of BMI cursor position with or without concurrent arm movement. We show in two monkeys that subtracting away estimated neural correlates of position improves online BMI performance only if the animals were allowed to move their arm. To understand why, we compared the neural variance attributable to cursor position when the same task was performed using arm reaching, versus arms-restrained BMI use. Firing rates correlated with both BMI cursor and hand positions, but hand positional effects were greater. To examine whether BMI position influences decoding in people with paralysis, we analyzed data from two intracortical BMI clinical trial participants and performed an online decoder comparison in one participant. We found only small motor cortical correlates, which did not affect performance. These results suggest that arm movement and proprioception are the major contributors to position-related motor cortical correlates. Cursor position visual feedback is therefore unlikely to affect the performance of BMI-driven prosthetic systems being developed for people with paralysis.

A fast intracortical brain–machine interface with patterned optogenetic feedback

Objective. The development of brain–machine interfaces (BMIs) brings new prospects to patients with a loss of autonomy. By combining online recordings of brain activity with a decoding algorithm, patients can learn to control a robotic arm in order to perform simple actions. However, in contrast to the vast amounts of somatosensory information channeled by limbs to the brain, current BMIs are devoid of touch and force sensors. Patients must therefore rely solely on vision and audition, which are maladapted to the control of a prosthesis. In contrast, in a healthy limb, somatosensory inputs alone can efficiently guide the handling of a fragile object, or ensure a smooth trajectory. We have developed a BMI in the mouse that includes a rich artificial somatosensory-like cortical feedback. Approach. Our setup includes online recordings of the activity of multiple neurons in the whisker primary motor cortex (vM1) and delivers feedback simultaneously via a low-latency, high-refresh-rate, spatially structured photo-stimulation of the whisker primary somatosensory cortex (vS1), based on a mapping obtained by intrinsic imaging. Main results. We demonstrate the operation of the loop and show that mice can detect the neuronal spiking in vS1 triggered by the photo-stimulations. Finally, we show that the mice can learn a behavioral task relying solely on the artificial inputs and outputs of the closed-loop BMI. Significance. This is the first motor BMI that includes a short-latency, intracortical, somatosensory-like feedback. It will be a useful platform to discover efficient cortical feedback schemes towards future human BMI applications.