Real-time Neural Signal Research Articles

Asynchronous Motor Imagery BCI and LiDAR-Based Shared Control System for Intuitive Wheelchair Navigation

Mapping drivers’ thoughts directly to mobility system control would make driving more intuitive as if the mobility system is an extension of their own body. Such a system would allow patients with motor disabilities to drive, as it would not require any physical movement. In this paper, we therefore propose a brain-controlled mobility system that analyzes real-time neural signals elicited from motor imagery, an imagination of different body movements. As such asynchronous brain-computer interfaces (BCIs) are prone to error, our system contains shared control capabilities that take into consideration continuously updated information of the surrounding environment along with electroencephalogram (EEG) signals to improve navigating performance without precise and accurate control from the driver. With our shared control method that uses a wheelchair with light detection and ranging (LiDAR) and inertial measurement unit (IMU) sensors, we held a comparative study in which participants drove our wheelchair with and without our shared control approach using either our brain-controlled system or a keyboard in a physical environment. The experimental results show that among the five participants, the three participants that failed the driving task with the asynchronous BCI-based system could also successfully complete it using our shared control approach. Furthermore, our approach narrows the gap between driving with neural signals and driving with a widely used interface in terms of both elapsed time and safety. These results show not only the potential of brain signals for driving, but also the applicability of BCIs to real-life situations.

A Multi-Channel Neural Recording System With Neural Spike Scan and Adaptive Electrode Selection for High-Density Neural Interface

There is an increasing demand for real-time neural signal monitoring from a large number of electrodes to provide adequate spatial and temporal resolution for high-density brain neural interfaces. This paper proposes an adaptive multi-channel neural recording system that can record neural signals from a large number of electrodes using a smaller number of recording channels. The system utilizes an adaptive electrode selection technique to automatically scan the electrodes where neural spikes occur and record those selected electrodes. A proposed 32-electrode neural recording prototype, including 12 recording channels and 8 scanning channels, was fabricated in a 180-nm CMOS process and tested <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in vitro</i> . With pre-recorded neural data, the system verified that the occurrence of neural spikes in specific electrodes could be detected and processed in real-time, enabling automatic electrode selection. Measured results showed that the proposed system could lead to over 40% reduction in silicon area compared to conventional works.

NCOG-14. INTRAOPERATIVE COGNITIVE-LINGUISTIC MAPPING GUIDED BY VISUALIZATION OF GAMMA BAND MODULATION ELECTROCORTICOGRAMS: PROOF OF CONCEPT IN A PATIENT WITH LEFT TEMPORAL AND OCCIPITAL LOW-GRADE ASTROCYTOMA

Abstract OBJECTIVE Determine the feasibility and preliminary utility of a novel approach to intraoperative brain mapping guided by visualization of electrocorticography (ECoG) heat maps. METHODS A 39-year-old male with a biopsy-proven left posterior temporal and occipital WHO grade II IDH-mutant astrocytoma underwent awake craniotomy with intraoperative language mapping. Language mapping utilized a dual iPad stimulus presentation system (NeuroMapper) coupled to a portable real-time neural signal processing system capable of both recording cortical activity and delivering direct cortical stimulation in a closed-loop fashion. An ECoG grid (4x8 with 1cm pitch) which covered the majority of the left temporal lobe was used to assess oscillatory cortical activity during administration of language paradigms including object, action, auditory descriptive, and written descriptive naming. ECoG recording and cortical stimulation were synchronized with stimulus presentation via a photosensor attached to the patient-facing tablet. Gamma band modulations in response to language paradigms at each electrode were processed in real-time and visualized as heat maps in MATLAB/Simulink. Following recording and visualization, bipolar direct cortical stimulation from the grid was conducted for each neighboring electrode pair (up to an intensity of 6 mA) during administration of language tasks. RESULTS Despite mild fluent aphasia, a large set of reliable baseline stimuli were obtained for the language mapping paradigms. All naming paradigms resulted in strongest heat map activation at electrode 12 located in the anterior to mid superior temporal gyrus. During stimulation, consistent speech arrest was observed across all paradigms when stimulating electrode pair 11-12, indicating good correspondence with ECoG heat map recordings. Additionally, this region corresponded well with posterior language network representation via resting-state fMRI. CONCLUSION Intraoperative real-time visualization of task-based ECoG gamma band modulation is feasible and may help identify targets for direct cortical stimulation. If validated, this may improve the efficiency and accuracy of intraoperative language mapping.

Algorithm and hardware considerations for real-time neural signal on-implant processing

Objective. Various on-workstation neural-spike-based brain machine interface (BMI) systems have reached the point of in-human trials, but on-node and on-implant BMI systems are still under exploration. Such systems are constrained by the area and battery. Researchers should consider the algorithm complexity, available resources, power budgets, CMOS technologies, and the choice of platforms when designing BMI systems. However, the effect of these factors is currently still unclear. Approaches. Here we have proposed a novel real-time 128 channel spike detection algorithm and optimised it on microcontroller (MCU) and field programmable gate array (FPGA) platforms towards consuming minimal power and memory/resources. It is presented as a use case to explore the different considerations in system design. Main results. The proposed spike detection algorithm achieved over 97% sensitivity and a smaller than 3% false detection rate. The MCU implementation occupies less than 3 KB RAM and consumes 31.5 µW ch−1. The FPGA platform only occupies 299 logic cells and 3 KB RAM for 128 channels and consumes 0.04 µW ch−1. Significance. On the spike detection algorithm front, we have eliminated the processing bottleneck by reducing the dynamic power consumption to lower than the hardware static power, without sacrificing detection performance. More importantly, we have explored the considerations in algorithm and hardware design with respect to scalability, portability, and costs. These findings can facilitate and guide the future development of real-time on-implant neural signal processing platforms.

A Multi-Channel Neural Recording System with Adaptive Electrode Selection for High-Density Neural Interface.

There is an increasing demand for real-time neural signal monitoring from a large number of electrode contacts to provide adequate spatial and temporal resolution for brain mapping and high-resolution neural interface. This paper proposes a novel multi-channel neural recording system that records neural signals from a large number of electrodes with a smaller number of recording channels. The system utilizes an adaptive electrode selection technique to automatically scan the electrode arrays and record from selected electrodes where neural spikes are detected. The proposed neural recording IC was fabricated in CMOS 180 nm process and tested with in vitro environments. Experiment results with pre-recorded neural data indicate that neural spikes can be separated and amplified with the proposed system and counted in real-time.

Real-Time Neural Signals of Disorder and Order Perception.

Order and disorder are prevalent in everyday life, yet little is known about the neural real-time processing that occurs during the perception of disorder relative to order. In the present study, from a cognitive perspective, by adopting the ERP method, we aimed to examine the elicited real-time neural signals of disorder and order perception when participants processed physical environmental and basic visual disorder and order pictures in an irrelevant red or green rectangle detection task, and we attempted to test the hypothesis of cognitive disfluency in disorder perception. Generally, we observed that at each measured time interval, the ERPs elicited by order stimuli were more positive (less negative) in amplitude than those elicited by disorder stimuli at the frontal electrodes (represented by F7/F8, FT7/FT8, Fz, and FCz), whereas at the posterior electrodes (represented by P7/P8, PO7/PO8, Pz, and POz), the opposite was true. These data reveal for the first time the neural underpinnings of disorder and order perception, extending our understanding of the nature of disorder and order. This study also contributes to the cognitive fluency literature and indirectly expands the research on disorder and order stimuli in cognitive fluency.

Neuromodulation targets pathological not physiological beta bursts during gait in Parkinson's disease

Freezing of gait (FOG) is a devastating axial motor symptom in Parkinson's disease (PD) leading to falls, institutionalization, and even death. The response of FOG to dopaminergic medication and deep brain stimulation (DBS) is complex, variable, and yet to be optimized. Fundamental gaps in the knowledge of the underlying neurobiomechanical mechanisms of FOG render this symptom one of the unsolved challenges in the treatment of PD. Subcortical neural mechanisms of gait impairment and FOG in PD are largely unknown due to the challenge of accessing deep brain circuitry and measuring neural signals in real time in freely-moving subjects. Additionally, there is a lack of gait tasks that reliably elicit FOG. Since FOG is episodic, we hypothesized that dynamic features of subthalamic (STN) beta oscillations, or beta bursts, may contribute to the Freezer phenotype in PD during gait tasks that elicit FOG. We also investigated whether STN DBS at 60 Hz or 140 Hz affected beta burst dynamics and gait impairment differently in Freezers and Non-Freezers. Synchronized STN local field potentials, from an implanted, sensing neurostimulator (Activa® PC + S, Medtronic, Inc.), and gait kinematics were recorded in 12 PD subjects, off-medication during forward walking and stepping-in-place tasks under the following randomly presented conditions: NO, 60 Hz, and 140 Hz DBS. Prolonged movement band beta burst durations differentiated Freezers from Non-Freezers, were a pathological neural feature of FOG and were shortened during DBS which improved gait. Normal gait parameters, accompanied by shorter bursts in Non-Freezers, were unchanged during DBS. The difference between the mean burst duration between hemispheres (STNs) of all individuals strongly correlated with the difference in stride time between their legs but there was no correlation between mean burst duration of each STN and stride time of the contralateral leg, suggesting an interaction between hemispheres influences gait. These results suggest that prolonged STN beta burst durations measured during gait is an important biomarker for FOG and that STN DBS modulated long not short burst durations, thereby acting to restore physiological sensorimotor information processing, while improving gait.

An Embedded Real-Time Processing Platform for Optogenetic Neuroprosthetic Applications.

Optogenetics offers a powerful new approach for controlling neural circuits. It has numerous applications in both basic and clinical science. These applications require stimulating devices with small processors that can perform real-time neural signal processing, deliver high-intensity light with high spatial and temporal resolution, and do not consume a lot of power. In this paper, we demonstrate the implementation of neuronal models in a platform consisting of an embedded system module and a portable digital light processing projector. As a replacement for damaged neural circuitry, the embedded module processes neural signals and then directs the projector to optogenetically activate a downstream neural pathway. We present a design in the context of stimulating circuits in the visual system, but the approach is feasible for a broad range of biomedical applications.

A real-time spike sorting method based on the embedded GPU.

Microelectrode arrays with hundreds of channels have been widely used to acquire neuron population signals in neuroscience studies. Online spike sorting is becoming one of the most important challenges for high-throughput neural signal acquisition systems. Graphic processing unit (GPU) with high parallel computing capability might provide an alternative solution for increasing real-time computational demands on spike sorting. This study reported a method of real-time spike sorting through computing unified device architecture (CUDA) which was implemented on an embedded GPU (NVIDIA JETSON Tegra K1, TK1). The sorting approach is based on the principal component analysis (PCA) and K-means. By analyzing the parallelism of each process, the method was further optimized in the thread memory model of GPU. Our results showed that the GPU-based classifier on TK1 is 37.92 times faster than the MATLAB-based classifier on PC while their accuracies were the same with each other. The high-performance computing features of embedded GPU demonstrated in our studies suggested that the embedded GPU provide a promising platform for the real-time neural signal processing.

Real-Time Neural Signal Sensing and Spike Sorting System Using Modified Zero-Crossing Feature with Highly Efficient Data Computation and Transmission

Real-Time Neural Signal Sensing and Spike Sorting System Using Modified Zero-Crossing Feature with Highly Efficient Data Computation and Transmission

Massively Parallel ECoG Signal Processing using MPSoC for Real-time Brain–Computer Interface

Bio-medical signal contains large quantities of data and it should be processed in a short period of time face. The development of modern computer based processing system provides extreme support for bio-medical signal analysis. It is a challenging work to process all the data recorded from an electrode array with high channel counts and bandwidth, such as electroencephalographic (EEG) and electrocorticographic (ECoG) grids. Therefore, a novel parallel processing method was developed for real-time neural signal processing of a brain–computer interface (BCI). The Multiprocessor system-on-chip (MPSoC) system was used to offload processing with the support of multi-core technology, which is capable of running many operations in parallel. The proposed design supports reconfigurable embedded computing techniques which can be portable, economical and cost saving system. The BCI system traces many channels of data. It may be processed and translated into a control signal, such as the movement of a cursor towards the indicated direction. This signal processing chain has three steps which are matrix–matrix multiplication technique, power spectral density calculation on every channel and classification of appropriate features for control. In this study, the BCI signal processing chain is implemented on the proposed embedded computing based SoC system, and compared with the current BCI system in terms of speed and accuracy. Significant performance gains were obtained in this study. The current BCI system processed 1000 channels of 250 ms in 1486 ms with 68% accuracy, while the proposed method took only 48 ms, an improvement of nearly 31 times.

Real-Time neural signal decoding on heterogeneous MPSocs based on VLIW ASIPs

An important research problem, at the basis of the development of embedded systems for neuroprosthetic applications, is the development of algorithms and platforms able to extract the patients motion intention by decoding the information encoded in neural signals. At the state of the art, no portable and reliable integrated solutions implementing such a decoding task have been identified. To this aim, in this paper, we investigate the possibility of using the MPSoC paradigm in this application domain. We perform a design space exploration that compares different custom MPSoC embedded architectures, implementing two versions of a on-line neural signal decoding algorithm, respectively targeting decoding of single and multiple acquisition channels. Each considered design points features a different application configuration, with a specific partitioning and mapping of parallel software tasks, executed on customized VLIW ASIP processing cores. Experimental results, obtained by means of FPGA-based prototyping and post-floorplanning power evaluation on a 40nm technology library, assess the performance and hardware-related costs of the considered configurations. The reported power figures demonstrate the usability of the MPSoC paradigm within the processing of bio-electrical signals and show the benefits achievable by the exploitation of the instruction-level parallelism within tasks.

Real-Time Neural Signals Decoding onto Off-the-Shelf DSP Processors for Neuroprosthetic Applications.

The control of upper limb neuroprostheses through the peripheral nervous system (PNS) can allow restoring motor functions in amputees. At present, the important aspect of the real-time implementation of neural decoding algorithms on embedded systems has been often overlooked, notwithstanding the impact that limited hardware resources have on the efficiency/effectiveness of any given algorithm. Present study is addressing the optimization of a template matching based algorithm for PNS signals decoding that is a milestone for its real-time, full implementation onto a floating-point digital signal processor (DSP). The proposed optimized real-time algorithm achieves up to 96% of correct classification on real PNS signals acquired through LIFE electrodes on animals, and can correctly sort spikes of a synthetic cortical dataset with sufficiently uncorrelated spike morphologies (93% average correct classification) comparably to the results obtained with top spike sorter (94% on average on the same dataset). The power consumption enables more than 24 h processing at the maximum load, and latency model has been derived to enable a fair performance assessment. The final embodiment demonstrates the real-time performance onto a low-power off-the-shelf DSP, opening to experiments exploiting the efferent signals to control a motor neuroprosthesis.

MPSoCs for real-time neural signal decoding: A low-power ASIP-based implementation

In this paper we target the design of a dedicated low-power computing platform for neuroprosthetic applications. The system must be capable of decoding the information encoded in neural signals, to extract the patients’ motion intention. To this aim, a highly-portable and reliable integrated processing device is required. However, a commonly acknowledged design methodology, to be used in such kind of design cases, is still not available in literature. In this work, we propose and assess the adoption of the MPSoC paradigm as a prospective solution. We present a design-case of a custom MPSoC integrated solution, implementing an on-line neural signal decoding algorithm. The proposed system executes parallel software tasks onto customized ASIP processing cores. Experimental results, obtained by placement- and activity-aware power evaluations carried out using an industrial 40 nm technology node as a reference, assess that the performance and power-related features of the designed architecture are compliant with the implantability constraints and with the battery lifetime required for real-life use. Moreover, besides the effectiveness of the proposed solution, this paper demonstrates also that custom heterogeneous MPSoCs can successfully challenge ultra-low power bio-medical signal processing problem.

A Custom dual-processor System for Real-time Neural Signal Processing

Abstract: This paper presents a custom dual-processor SoC architecture, studied and customized to support information extraction from signals acquired from Peripheral Neural System, for prosthetic applications. The main tasks accomplished by the processing implemented on the computing platform are noise removal and identification of neural spikes. On-board execution of such tasks allows to identify which samples actually contain useful information. Thus, it reduces required input/output bandwidth, so that connection to the external environment can be implemented using a Bluetooth Low Energy device. The overall SoC architecture has power consumption compliant with implant-related constraints with a battery lifetime of around one-day.

A Custom MPSoC Architecture With Integrated Power Management for Real-Time Neural Signal Decoding

Bioengineering research is posing hard challenges to digital embedded system designers. Tight real-time constraints, miniaturization, and low power are critical issues exacerbated by applications requiring the implant of electronic devices in the patient's body. Among them, neurocontrolled motor prostheses are on the cutting edge of the research in the field, requiring the real-time neural signal decoding to extract the patient's movement intention in order to control the mechatronic device. Despite the literature in the field, how to implement a highly-portable and reliable integrated platform is still an open question. In this paper, we propose a field-programmable gate array-based prototype of an multi-processor system-on-chip embedded architecture that implements an online neural signal decoding algorithm. The prototype is capable of respecting the real-time constraints posed by the application when clocked at less than 50 MHz. Considering that the application workload is extremely data dependent and unpredictable, the architecture has to be dimensioned taking into account critical worst-case operating conditions to ensure robustness. To compensate the resulting over-provisioning of the system architecture, a software-controllable power management has been integrated. Experimental results demonstrate the real-time behavior and allow evaluating the usefulness of the proposed power management technique on public databases.

Improving brain–machine interface performance by decoding intended future movements

Objective. A brain–machine interface (BMI) records neural signals in real time from a subject's brain, interprets them as motor commands, and reroutes them to a device such as a robotic arm, so as to restore lost motor function. Our objective here is to improve BMI performance by minimizing the deleterious effects of delay in the BMI control loop. We mitigate the effects of delay by decoding the subject's intended movements a short time lead in the future. Approach. We use the decoded, intended future movements of the subject as the control signal that drives the movement of our BMI. This should allow the user's intended trajectory to be implemented more quickly by the BMI, reducing the amount of delay in the system. In our experiment, a monkey (Macaca mulatta) uses a future prediction BMI to control a simulated arm to hit targets on a screen. Main Results. Results from experiments with BMIs possessing different system delays (100, 200 and 300 ms) show that the monkey can make significantly straighter, faster and smoother movements when the decoder predicts the user's future intent. We also characterize how BMI performance changes as a function of delay, and explore offline how the accuracy of future prediction decoders varies at different time leads. Significance. This study is the first to characterize the effects of control delays in a BMI and to show that decoding the user's future intent can compensate for the negative effect of control delay on BMI performance.

A Wideband Dual-Antenna Receiver for Wireless Recording From Animals Behaving in Large Arenas

A low-noise wideband receiver (Rx) is presented for a multichannel wireless implantable neural recording (WINeR) system that utilizes time-division multiplexing of pulse width modulated (PWM) samples. The WINeR-6 Rx consists of four parts: 1) RF front end; 2) signal conditioning; 3) analog output (AO); and 4) field-programmable gate array (FPGA) back end. The RF front end receives RF-modulated neural signals in the 403-490MHz band with a wide bandwidth of 18MHz. The frequency-shift keying (FSK) PWM demodulator in the FPGA is a time-to-digital converter with 304ps resolution, which converts the analog pulse width information to 16-bit digital samples. Automated frequency tracking has been implemented in the Rx to lock onto the free-running voltage-controlled oscillator in the transmitter (Tx). Two antennas and two parallel RF paths are used to increase the wireless coverage area. BCI-2000 graphical user interface has been adopted and modified to acquire, visualize, and record the recovered neural signals in real time. The AO module picks three demultiplexed channels and converts them into analog signals for direct observation on an oscilloscope. One of these signals is further amplified to generate an audio output, offering users the ability to listen to ongoing neural activity. Bench-top testing of the Rx performance with a 32-channel WINeR-6 Tx showed that the input referred noise of the entire system at a Tx-Rx distance of 1.5m was 4.58μV rms with 8-bit resolution at 640 kSps. In an in vivo experiment, location-specific receptive fields of hippocampal place cells were mapped during a behavioral experiment in which a rat completed 40 laps in a large circular track. Results were compared against those acquired from the same animal and the same set of electrodes by a commercial hardwired recording system to validate the wirelessly recorded signals.

Cortical Adaptation to a Chronic Micro-Electrocorticographic Brain Computer Interface

Brain-computer interface (BCI) technology decodes neural signals in real time to control external devices. In this study, chronic epidural micro-electrocorticographic recordings were performed over primary motor (M1) and dorsal premotor (PMd) cortex of three macaque monkeys. The differential gamma-band amplitude (75-105 Hz) from two arbitrarily chosen 300 μm electrodes (one located over each cortical area) was used for closed-loop control of a one-dimensional BCI device. Each monkey rapidly learned over a period of days to successfully control the velocity of a computer cursor. While both cortical areas contributed to success on the BCI task, the control signals from M1 were consistently modulated more strongly than those from PMd. Additionally, we observe that gamma-band power during active BCI control is always above resting brain activity. This suggests that purposeful gamma-band modulation is an active process that is obtained through increased cortical activation.

Bright ideas

Sensing neuronal activity using fluorescence has many potential advantages over current methods. Now, by taking advantage of photoinduced electron transfer, fluorescent sensors have been developed that allow high-fidelity recording of neural signals in real time.