Online Brain Machine Interface Research Articles

A spiking neural network with continuous local learning for robust online brain machine interface

Objective. Spiking neural networks (SNNs) are powerful tools that are well suited for brain machine interfaces (BMI) due to their similarity to biological neural systems and computational efficiency. They have shown comparable accuracy to state-of-the-art methods, but current training methods require large amounts of memory, and they cannot be trained on a continuous input stream without pausing periodically to perform backpropagation. An ideal BMI should be capable training continuously without interruption to minimize disruption to the user and adapt to changing neural environments. Approach. We propose a continuous SNN weight update algorithm that can be trained to perform regression learning with no need for storing past spiking events in memory. As a result, the amount of memory needed for training is constant regardless of the input duration. We evaluate the accuracy of the network on recordings of neural data taken from the premotor cortex of a primate performing reaching tasks. Additionally, we evaluate the SNN in a simulated closed loop environment and observe its ability to adapt to sudden changes in the input neural structure. Main results. The continuous learning SNN achieves the same peak correlation () as existing SNN training methods when trained offline on real neural data while reducing the total memory usage by 92%. Additionally, it matches state-of-the-art accuracy in a closed loop environment, demonstrates adaptability when subjected to multiple types of neural input disruptions, and is capable of being trained online without any prior offline training. Significance. This work presents a neural decoding algorithm that can be trained rapidly in a closed loop setting. The algorithm increases the speed of acclimating a new user to the system and also can adapt to sudden changes in neural behavior with minimal disruption to the user.

Short-training Algorithm for Online Brain-machine Interfaces Using One-photon Microendoscopic Calcium Imaging.

Calcium imaging has great potential to be applied to online brain-machine interfaces (BMIs). As opposed to two-photon imaging settings, a one-photon microendoscopic imaging device can be chronically implanted and is subject to little motion artifacts. Traditionally, one-photon microendoscopic calcium imaging data are processed using the constrained nonnegative matrix factorization (CNMFe) algorithm, but this batched processing algorithm cannot be applied in real-time. An online analysis of calcium imaging data algorithm (or OnACIDe) has been proposed, but OnACIDe updates the neural components by repeatedly performing neuron identification frame-by-frame, which may decelerate the update speed if applying to online BMIs. For BMI applications, the ability to track a stable population of neurons in real-time has a higher priority over accurately identifying all the neurons in the field of view. By leveraging the fact that 1) microendoscopic recordings are rather stable with little motion artifacts and 2) the number of neurons identified in a short training period is sufficient for potential online BMI tasks such as cursor movements, we proposed the short-training CNMFe algorithm (stCNMFe) that skips motion correction and neuron identification processes to enable a more efficient BMI training program in a one-photon microendoscopic setting.

EEG model stability and online decoding of attentional demand during gait using gamma band features

Rehabilitation therapies are evolving oriented to improve their performances in terms of functional recovery. To achieve such recovery, the patients’ involvement is an important factor that correlates with the plastic properties of the brain. By evaluating electroencephalographic signals, it is possible to modify, in real time, the parameters of the rehabilitation according to the patients’ cognitive state. In this paper, an online brain–machine interface to measure the attention level during gait is presented. The system is based on the measurement of selective attention mechanisms manifested as power synchronization and desynchronization in the gamma band. A Linear Discriminant Analysis classifier is used to provide an attention index between 0 and 1 in real time. Robust techniques for artifact rejection and signal standardization are used in order to deal with the problems associated to the measurement of cortical signals during walking. The final interface is validated with 4 incomplete Spinal Cord Injury patients and 4 healthy participants. The system shows an average success rate of 68.1% in the classification of 3 attention levels and a stable behavior of these results during time.

Brain-Actuated Control of Dual-Arm Robot Manipulation With Relative Motion

This paper describes the brain-actuated control of a dual-arm robot performing bimanual relative motion manipulation tasks, through the adoption of relative Jacobian matrix, wherein the dual-arm robot can be considered as a single manipulator and the movements of the end effectors can be calculated by the relative motion. An online brain–machine interface (BMI) system based on multichannel steady-state visual evoked potentials was developed, and it was able to perform band-pass filtering and visual stimuli classification using support vector machines. Considering the relative motion in a constrained plane, the asymmetric bimanual manipulation can be transformed into 2-D control tasks through polar coordinate transformation such that the end effectors can achieve smooth direction-and-distance motion in the arbitrary position of the operational space. Moreover, the kinematics redundancy scheme, using online neuro-dynamics optimization, was developed for joint velocity optimization subject to physical constraints. Five individuals participated in the experiments and successfully fulfilled the given manipulation task.

A Self-Adaptive Online Brain–Machine Interface of a Humanoid Robot Through a General Type-2 Fuzzy Inference System

This paper presents a self-adaptive autonomous online learning through a general type-2 fuzzy system (GT2 FS) for the motor imagery (MI) decoding of a brain-machine interface (BMI) and navigation of a bipedal humanoid robot in a real experiment, using electroencephalography (EEG) brain recordings only. GT2 FSs are applied to BMI for the first time in this study. We also account for several constraints commonly associated with BMI in real practice: 1) the maximum number of EEG channels is limited and fixed; 2) no possibility of performing repeated user training sessions; and 3) desirable use of unsupervised and low-complexity feature extraction methods. The novel online learning method presented in this paper consists of a self-adaptive GT2 FS that can autonomously self-adapt both its parameters and structure via creation, fusion, and scaling of the fuzzy system rules in an online BMI experiment with a real robot. The structure identification is based on an online GT2 Gath–Geva algorithm where every MI decoding class can be represented by multiple fuzzy rules (models), which are learnt in a continous (trial-by-trial) non-iterative basis. The effectiveness of the proposed method is demonstrated in a detailed BMI experiment, in which 15 untrained users were able to accurately interface with a humanoid robot, in a single session, using signals from six EEG electrodes only.

Real-time particle filtering and smoothing algorithms for detecting abrupt changes in neural ensemble spike activity.

Sequential change-point detection from time series data is a common problem in many neuroscience applications, such as seizure detection, anomaly detection, and pain detection. In our previous work (Chen Z, Zhang Q, Tong AP, Manders TR, Wang J. J Neural Eng 14: 036023, 2017), we developed a latent state-space model, known as the Poisson linear dynamical system, for detecting abrupt changes in neuronal ensemble spike activity. In online brain-machine interface (BMI) applications, a recursive filtering algorithm is used to track the changes in the latent variable. However, previous methods have been restricted to Gaussian dynamical noise and have used Gaussian approximation for the Poisson likelihood. To improve the detection speed, we introduce non-Gaussian dynamical noise for modeling a stochastic jump process in the latent state space. To efficiently estimate the state posterior that accommodates non-Gaussian noise and non-Gaussian likelihood, we propose particle filtering and smoothing algorithms for the change-point detection problem. To speed up the computation, we implement the proposed particle filtering algorithms using advanced graphics processing unit computing technology. We validate our algorithms, using both computer simulations and experimental data for acute pain detection. Finally, we discuss several important practical issues in the context of real-time closed-loop BMI applications. NEW & NOTEWORTHY Sequential change-point detection is an important problem in closed-loop neuroscience experiments. This study proposes novel sequential Monte Carlo methods to quickly detect the onset and offset of a stochastic jump process that drives the population spike activity. This new approach is robust with respect to spike sorting noise and varying levels of signal-to-noise ratio. The GPU implementation of the computational algorithm allows for parallel processing in real time.

Design and Analysis of Closed-Loop Decoder Adaptation Algorithms for Brain-Machine Interfaces

Closed-loop decoder adaptation (CLDA) is an emerging paradigm for achieving rapid performance improvements in online brain-machine interface (BMI) operation. Designing an effective CLDA algorithm requires making multiple important decisions, including choosing the timescale of adaptation, selecting which decoder parameters to adapt, crafting the corresponding update rules, and designing CLDA parameters. These design choices, combined with the specific settings of CLDA parameters, will directly affect the algorithm's ability to make decoder parameters converge to values that optimize performance. In this article, we present a general framework for the design and analysis of CLDA algorithms and support our results with experimental data of two monkeys performing a BMI task. First, we analyze and compare existing CLDA algorithms to highlight the importance of four critical design elements: the adaptation timescale, selective parameter adaptation, smooth decoder updates, and intuitive CLDA parameters. Second, we introduce mathematical convergence analysis using measures such as mean-squared error and KL divergence as a useful paradigm for evaluating the convergence properties of a prototype CLDA algorithm before experimental testing. By applying these measures to an existing CLDA algorithm, we demonstrate that our convergence analysis is an effective analytical tool that can ultimately inform and improve the design of CLDA algorithms.

An online brain–machine interface using decoding of movement direction from the human electrocorticogram

A brain–machine interface (BMI) can be used to control movements of an artificial effector, e.g. movements of an arm prosthesis, by motor cortical signals that control the equivalent movements of the corresponding body part, e.g. arm movements. This approach has been successfully applied in monkeys and humans by accurately extracting parameters of movements from the spiking activity of multiple single neurons. We show that the same approach can be realized using brain activity measured directly from the surface of the human cortex using electrocorticography (ECoG). Five subjects, implanted with ECoG implants for the purpose of epilepsy assessment, took part in our study. Subjects used directionally dependent ECoG signals, recorded during active movements of a single arm, to control a computer cursor in one out of two directions. Significant BMI control was achieved in four out of five subjects with correct directional decoding in 69%–86% of the trials (75% on average). Our results demonstrate the feasibility of an online BMI using decoding of movement direction from human ECoG signals. Thus, to achieve such BMIs, ECoG signals might be used in conjunction with or as an alternative to intracortical neural signals.

ERD-Based Online Brain–Machine Interfaces (BMI) in the Context of Neurorehabilitation: Optimizing BMI Learning and Performance

Event-related desynchronization (ERD) of sensori-motor rhythms (SMR) can be used for online brain-machine interface (BMI) control, but yields challenges related to the stability of ERD and feedback strategy to optimize BMI learning.Here, we compared two approaches to this challenge in 20 right-handed healthy subjects (HS, five sessions each, S1-S5) and four stroke patients (SP, 15 sessions each, S1-S15). ERD was recorded from a 275-sensor MEG system. During daily training,motor imagery-induced ERD led to visual and proprioceptive feedback delivered through an orthotic device attached to the subjects' hand and fingers. Group A trained with a heterogeneous reference value (RV) for ERD detection with binary feedback and Group B with a homogenous RV and graded feedback (10 HS and 2 SP in each group). HS in Group B showed better BMI performance than Group A (p < 0.001) and improved BMI control from S1 to S5 (p = 0.012) while Group A did not. In spite of the small n, SP in Group B showed a trend for a higher BMI performance (p = 0.06) and learning was significantly better (p < 0.05). Using a homogeneous RV and graded feedback led to improved modulation of ipsilesional activity resulting in superior BMI learning relative to use of a heterogeneous RV and binary feedback.

An Online Brain-Machine Interface Using Decoding Of Movement Direction From The Human Electrocorticogram

Event Abstract Back to Event An Online Brain-Machine Interface Using Decoding Of Movement Direction From The Human Electrocorticogram Tomislav Milekovic1, 2*, Joerg Fischer2, Tonio Ball1, 3, Andreas Schulze-Bonhage1, 3, Ad Aertsen1, 4 and Carsten Mehring1, 2 1 Albert-Ludwigs-University Freiburg, Bernstein Center for Computational Neuroscience, Germany 2 Albert-Ludwigs-University Freiburg, Institute for Biology I, Germany 3 University Hospital Freiburg, Epilepsy Center, Germany 4 Albert-Ludwigs-University Freiburg, Institute for Biology III, Germany Brain-machine interfaces (BMIs) can be characterized by the approach used to translate brain signals into effector movements. Here we use a “direct motor” BMI approach where movements of an artificial effector (e.g. movement of an arm prosthesis to the right) are controlled by motor cortical signals that control the equivalent movements of the corresponding body part (e.g. arm movement to the right). This approach has been successfully applied in monkeys and humans by accurately extracting parameters of movements from the spiking activity of multiple single-units. Here we show that the same approach can be realized using brain activity measured directly at the surface of the human cortex (electrocorticogram, ECoG).Three subjects suffering from intractable pharmaco-resistant epilepsy voluntarily participated in the study after having given their informed consent (study approved by the Freiburg University Hospital's Ethics Committee). As a part of pre-surgical diagnosis all subjects had 8x8 ECoG grid implants (4 mm electrode diameter, 10 mm inter-electrode distance, Ad-Tech Medical Instruments, USA) over the hand and arm motor cortex. Subjects interacted with an experimental paradigm shown on a computer screen. Each trial consisted of a pause phase (1-2 sec) followed by a preparatory informative cue (1-2 sec) which informed the subject to prepare for executing or imagining a hand/arm movement to the left or to the right using the hand contralateral to the implantation site. After a delay of 2-3 sec, a go cue was presented and subjects had to perform the movement execution or imagination within the next two seconds. Subsequently, a cursor on the screen was moved according to the movement direction decoded from the subjects’ ECoG signals. Closed loop BMI control of movement direction was realized using low-pass filtered (symmetric Savitzky-Golay filter, 2nd order, between 0.25 and 1 sec window length) ECoG signals during movement execution or movement imagination. For movement execution significant BMI control was achieved for all three subjects in all 7 sessions with correct directional decoding in 69%-86% of the trials (79% on average across all sessions). Movement imagination was carried out with only one subject where 3 out of 4 sessions showed significant BMI control with correct decoding in 66%-72% of the trials (69% on average).In summary, our results demonstrate the principle feasibility of an online direct motor BMI using ECoG signals. Thus, for a direct motor BMI, ECoG might be used in conjunction or as an alternative to the intra-cortical neural signals, with possible advantages due to reduced invasiveness. Acknowledgements Work supported by BMBF 01GQ0420 to BCCN Freiburg and BMBF GoBio grant 0313891. Keywords: computational neuroscience Conference: Bernstein Conference on Computational Neuroscience, Berlin, Germany, 27 Sep - 1 Oct, 2010. Presentation Type: Presentation Topic: Bernstein Conference on Computational Neuroscience Citation: Milekovic T, Fischer J, Ball T, Schulze-Bonhage A, Aertsen A and Mehring C (2010). An Online Brain-Machine Interface Using Decoding Of Movement Direction From The Human Electrocorticogram. Front. Comput. Neurosci. Conference Abstract: Bernstein Conference on Computational Neuroscience. doi: 10.3389/conf.fncom.2010.51.00028 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 20 Sep 2010; Published Online: 23 Sep 2010. * Correspondence: Dr. Tomislav Milekovic, Albert-Ludwigs-University Freiburg, Bernstein Center for Computational Neuroscience, Freiburg, Germany, milekovic@bccn.uni-freiburg.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Tomislav Milekovic Joerg Fischer Tonio Ball Andreas Schulze-Bonhage Ad Aertsen Carsten Mehring Google Tomislav Milekovic Joerg Fischer Tonio Ball Andreas Schulze-Bonhage Ad Aertsen Carsten Mehring Google Scholar Tomislav Milekovic Joerg Fischer Tonio Ball Andreas Schulze-Bonhage Ad Aertsen Carsten Mehring PubMed Tomislav Milekovic Joerg Fischer Tonio Ball Andreas Schulze-Bonhage Ad Aertsen Carsten Mehring Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.