Neural Decoding Research Articles

Accelerating Neural BP-Based Decoder Using Coded Distributed Computing

Accelerating Neural BP-Based Decoder Using Coded Distributed Computing

A high‐performance asynchronous readout circuit with cascade function for a neural recording micro‐electrode array

AbstractThe data‐driven machine learning technology used for neural decoding emphasizes the requirements of in vivo and in vitro neural signal acquisition with high spatial and temporal resolution. However, micro‐electrode arrays (MEAs) that simultaneously achieve high spatial and temporal resolution neural signal acquisition are not yet available. Meanwhile, the high data bandwidth of large‐scale MEA brings challenges in power consumption, data transmission, storage, and neural signal processing. This research aims to improve the spatial and temporal resolution of MEA, reduce the cost and time of large‐scale MEA customization through cascading, and reduce the bandwidth of large‐scale MEA through spike compression. Firstly, based on in‐pixel spike detection, a row‐based neural spike readout mechanism and related array circuit to improve the neural spike readout performance and temporal resolution of neural signal acquisition is proposed. To further enhance the spatial resolution and reduce the risk of large‐scale MEA fabrication, the cascading capability of the readout circuit is explored. Lastly, spatial correlation‐based neural spike encoding is proposed to reduce the data bandwidth, achieving a 5.2× compression rate. This is a study on implementing large‐scale MEA through cascaded readout circuits and novel study to utilize the spatial correlation between detected neural spikes for further compression.

Decoding reveals the neural representation of perceived and imagined musical sounds.

Vividly imagining a song or a melody is a skill that many people accomplish with relatively little effort. However, we are only beginning to understand how the brain represents, holds, and manipulates these musical "thoughts". Here, we decoded perceived and imagined melodies from magnetoencephalography (MEG) brain data (N = 71) to characterize their neural representation. We found that, during perception, auditory regions represent the sensory properties of individual sounds. In contrast, a widespread network including fronto-parietal cortex, hippocampus, basal nuclei, and sensorimotor regions hold the melody as an abstract unit during both perception and imagination. Furthermore, the mental manipulation of a melody systematically changes its neural representation, reflecting volitional control of auditory images. Our work sheds light on the nature and dynamics of auditory representations, informing future research on neural decoding of auditory imagination.

Optogenetic Brain-Computer Interfaces.

The brain-computer interface (BCI) is one of the most powerful tools in neuroscience and generally includes a recording system, a processor system, and a stimulation system. Optogenetics has the advantages of bidirectional regulation, high spatiotemporal resolution, and cell-specific regulation, which expands the application scenarios of BCIs. In recent years, optogenetic BCIs have become widely used in the lab with the development of materials and software. The systems were designed to be more integrated, lightweight, biocompatible, and power efficient, as were the wireless transmission and chip-level embedded BCIs. The software is also constantly improving, with better real-time performance and accuracy and lower power consumption. On the other hand, as a cutting-edge technology spanning multidisciplinary fields including molecular biology, neuroscience, material engineering, and information processing, optogenetic BCIs have great application potential in neural decoding, enhancing brain function, and treating neural diseases. Here, we review the development and application of optogenetic BCIs. In the future, combined with other functional imaging techniques such as near-infrared spectroscopy (fNIRS) and functional magnetic resonance imaging (fMRI), optogenetic BCIs can modulate the function of specific circuits, facilitate neurological rehabilitation, assist perception, establish a brain-to-brain interface, and be applied in wider application scenarios.

Neural Decoding of Spontaneous Overt and Intended Speech.

The aim of this study was to decode intended and overt speech from neuromagnetic signals while the participants performed spontaneous overt speech tasks without cues or prompts (stimuli). Magnetoencephalography (MEG), a noninvasive neuroimaging technique, was used to collect neural signals from seven healthy adult English speakers performing spontaneous, overt speech tasks. The participants randomly spoke the words yes or no at a self-paced rate without cues. Two machine learning models, namely, linear discriminant analysis (LDA) and one-dimensional convolutional neural network (1D CNN), were employed to classify the two words from the recorded MEG signals. LDA and 1D CNN achieved average decoding accuracies of 79.02% and 90.40%, respectively, in decoding overt speech, significantly surpassing the chance level (50%). The accuracy for decoding intended speech was 67.19% using 1D CNN. This study showcases the possibility of decoding spontaneous overt and intended speech directly from neural signals in the absence of perceptual interference. We believe that these findings make a steady step toward the future spontaneous speech-based brain-computer interface.

Subword Representations Successfully Decode Brain Responses to Morphologically Complex Written Words

Abstract This study extends the idea of decoding word-evoked brain activations using a corpus-semantic vector space to multimorphemic words in the agglutinative Finnish language. The corpus-semantic models are trained on word segments, and decoding is carried out with word vectors that are composed of these segments. We tested several alternative vector-space models using different segmentations: no segmentation (whole word), linguistic morphemes, statistical morphemes, random segmentation, and character-level 1-, 2- and 3-grams, and paired them with recorded MEG responses to multimorphemic words in a visual word recognition task. For all variants, the decoding accuracy exceeded the standard word-label permutation-based significance thresholds at 350–500 ms after stimulus onset. However, the critical segment-label permutation test revealed that only those segmentations that were morphologically aware reached significance in the brain decoding task. The results suggest that both whole-word forms and morphemes are represented in the brain and show that neural decoding using corpus-semantic word representations derived from compositional subword segments is applicable also for multimorphemic word forms. This is especially relevant for languages with complex morphology, because a large proportion of word forms are rare and it can be difficult to find statistically reliable surface representations for them in any large corpus.

ADVANCEMENTS IN BRAIN-COMPUTER INTERFACES: THE IMPACT OF AI AND GENERATIVE MODELS ON NEURAL DECODING AND APPLICATION

Brain-computer interfaces (BCIs) enable direct communication between the brain and external devices, offering transformative potential for individuals with neurological disorders and beyond. Recent advancements in artificial intelligence (AI), particularly in generative models, have significantly improved the accuracy and efficiency of neural decoding in BCIs. This paper provides a comprehensive overview of these advancements, focusing on the role of AI and generative models in enhancing BCI performance. I discuss the contributions of leading companies like Neuralink, Synchron, and Paradromics, and analyze key research findings on signal processing, machine learning algorithms, and ethical considerations. My synthesis of current literature and industry insights highlights the transformative potential of AI-driven BCIs while addressing the technical and ethical challenges that need to be overcome to fully realize their potential. I emphasize the importance of continued research in refining generative AI models, enhancing real-time processing capabilities, and establishing ethical frameworks to guide the development and application of BCIs.

A Neural Network-Based Compressive LDPC Decoder Design Over Correlated Noise Channel

A Neural Network-Based Compressive LDPC Decoder Design Over Correlated Noise Channel

Connectome-constrained neural decoding reveals a representational hierarchy from perception to cognition to action

Connectome-constrained neural decoding reveals a representational hierarchy from perception to cognition to action

Adaptive EMG decomposition in dynamic conditions based on online learning metrics with tunable hyperparameters

Objective. Developing neural decoders robust to non-stationary conditions is essential to ensure their long-term accuracy and stability. This is particularly important when decoding the neural drive to muscles during dynamic contractions, which pose significant challenges for stationary decoders. Approach. We propose a novel adaptive electromyography (EMG) decomposition algorithm that builds on blind source separation methods by leveraging the Kullback–Leibler divergence and kurtosis of the signals as metrics for online learning. The proposed approach provides a theoretical framework to tune the adaptation hyperparameters and compensate for non-stationarities in the mixing matrix, such as due to dynamic contractions, and to identify the underlying motor neuron (MN) discharges. The adaptation is performed in real-time (∼22 ms of computational time per 100 ms batches). Main results. The hyperparameters of the proposed adaptation captured anatomical differences between recording locations (forearm vs wrist) and generalised across subjects. Once optimised, the proposed adaptation algorithm significantly improved all decomposition performance metrics with respect to the absence of adaptation in a wide range of motion of the wrist (80∘). The rate of agreement, sensitivity, and precision were ⩾90% in ⩾80% of the cases in both simulated and experimentally recorded data, according to a two-source validation approach. Significance. The findings demonstrate the suitability of the proposed online learning metrics and hyperparameter optimisation to compensate the induced modulation and accurately decode MN discharges in dynamic conditions. Moreover, the study proposes an experimental validation method for EMG decomposition in dynamic tasks.

A novel binary quantizer for variational autoencoder-based image compressor

Neural network-based encoder and decoder are one of the emerging techniques for image compression. To improve the compression rate, these models use a special module called the quantizer that improves the entropy of the generated representation. Here, we propose a novel binary quantizer and an auxiliary support module known as the remapper that co-ordinate for better compression and decompression of an image. The proposed quantizer generates the compressed binary representation whereas the remapper aids in minimizing the reconstruction error during decompression. The combined optimization of the model consisting of the encoder, quantizer, remapper, and the decoder as a single entity is not feasible due to the non-differentiable nature of the quantizer. To mitigate this, we propose a step-by-step training process wherein the encoder and decoder are trained first, then keeping the encoder parameters fixed, the remapper is trained along with the decoder to obtain a fully optimized model. The training is performed with different types of images and the results are compared to understand the best training procedures. Experimental results reveal that the proposed model can compress a 256 × 256 × 3 image to a mere 256 bits, thereby achieving a compression ratio of 6144:1; with excellent perceptual reconstruction during decompression.

Discriminative Feature Enhancement Network for few-shot classification and beyond

Few-shot classification aims to recognize query samples from novel classes given scarce labeled data, which remains a challenging problem in machine learning. This paper proposes a Discriminative Feature Enhancement Network (DFENet) to distinguish the discriminative feature of a novel category with the following characteristics: (1) a Cross-Modal Guidance Module (CMG-Module) is proposed to enrich the query vectors via leveraging label information from the additional modality; (2) a Neural-Decoding based Attention Module (NDA-Module) further explores the relationship between the query and support samples, and the attention weights depend on contribution of the support to the reconstructed query. The main intuition of the NDA-Module is to deepen our understanding of the self-attention mechanism from a neural decoding perspective, emphasizing that the best reconstruction can be used as a universal guideline; (3) a flexible triplet loss is designed to incorporate the semantic context among all classes and distinguish samples from different classes. The experimental results on two few-shot classification datasets show that our DFENet can learn a more discriminative representation for novel classes progressively. We also test the proposed method on image retrieval and facial expression recognition tasks, demonstrating consistent improvement.

Extended Poisson Gaussian-Process Latent Variable Model for Unsupervised Neural Decoding.

Dimension reduction on neural activity paves a way for unsupervised neural decoding by dissociating the measurement of internal neural pattern reactivation from the measurement of external variable tuning. With assumptions only on the smoothness of latent dynamics and of internal tuning curves, the Poisson gaussian-process latent variable model (P-GPLVM; Wu etal., 2017) is a powerful tool to discover the low-dimensional latent structure for high-dimensional spike trains. However, when given novel neural data, the original model lacks a method to infer their latent trajectories in the learned latent space, limiting its ability for estimating the neural reactivation. Here, we extend the P-GPLVM to enable the latent variable inference of new data constrained by previously learned smoothness and mapping information. We also describe a principled approach for the constrained latent variable inference for temporally compressed patterns of activity, such as those found in population burst events during hippocampal sharp-wave ripples, as well as metrics for assessing the validity of neural pattern reactivation and inferring the encoded experience. Applying these approaches to hippocampal ensemble recordings during active maze exploration, we replicate the result that P-GPLVM learns a latent space encoding the animal's position. We further demonstrate that this latent space can differentiate one maze context from another. By inferring the latent variables of new neural data during running, certain neural patterns are observed to reactivate, in accordance with the similarity of experiences encoded by its nearby neural trajectories in the training data manifold. Finally, reactivation of neural patterns can be estimated for neural activity during population burst events as well, allowing the identification for replay events of versatile behaviors and more general experiences. Thus, our extension of the P-GPLVM framework for unsupervised analysis of neural activity can be used to answer critical questions related to scientific discovery.

A data compression algorithm with the improved SRLE for high-throughput neural signal acquisition device.

Multi-channel acquisition systems of brain neural signals can provide a powerful tool with a wide range of information for the clinical application of brain computer interfaces. High-throughput implantable systems are limited by size and power consumption, posing challenges to system design. To acquire more comprehensive neural signals and wirelessly transmit high-throughput brain neural signals, a FPGA-based acquisition system for multi-channel brain nerve signals has been developed. And the Bluetooth transmission with low-power technology are utilized. To wirelessly transmit large amount of data with limited Bluetooth bandwidth and improve the accuracy of neural signal decoding, an improved sharing run length encoding (SRLE) is proposed to compress the spike data of brain neural signal to improve the transmission efficiency of the system. The functional prototype has been developed, which consists of multi-channel data acquisition chips, FPGA main control module with the improved SRLE, a wireless data transmitter, a wireless data receiver and an upper computer. And the developed functional prototype was tested for spike detection of brain neural signal by animal experiments. From the animal experiments, it shows that the system can successfully collect and transmit brain nerve signals. And the improved SRLE algorithm has an excellent compression effect with the average compression rate of 5.94%, compared to the double run-length encoding, the FDR encoding, and the traditional run-length encoding. The developed system, incorporating the improved SRLE algorithm, is capable of wirelessly capturing spike signals with 1024 channels, thereby realizing the implantable systems of High-throughput brain neural signals.

High-performance decoding using deep neural networks

Problem statement. Deep neural networks are successfully used in channel coding to improve the quality of decoding. However, modern neural channel decoders cannot achieve high decoding performance and low complexity at the same time. To overcome this problem, it is necessary to develop a decoder with a double residual neuron. By integrating residual input and residual learning into the neural channel decoder architecture, it will significantly improve decoding performance while maintaining low complexity. Goal. Development of a decoder architecture with a dual residual neural network, which provides high decoding performance in channel coding with low model complexity and computational costs. Results. The simulation results show that the proposed decoder with a double residual neural network provides a significant increase in decoding performance. Compared to a neural decoder based on a trust propagation algorithm, a decoder with a dual residual neural network provides an additional 0.5-1.8 dB encoding gain compared to different channel codes. Compared to the hypergraph neural decoder, which has the best error correction capability among all existing neural channel decoders, the dual residual neural network decoder also provides similar or even better decoding performance for different channel codes. Practical significance. The results obtained can be used to improve decoding efficiency and improve performance in channel coding. The work was carried out with the financial support of the Ministry of Science and Higher Education of the Russian Federation within the framework of the state assignment "youth laboratory" No. FZGM-2024-0003.

Dynamics of odor-source localization: Insights from real-time odor plume recordings and head-motion tracking in freely moving mice.

Animals navigating turbulent odor plumes exhibit a rich variety of behaviors, and employ efficient strategies to locate odor sources. A growing body of literature has started to probe this complex task of localizing airborne odor sources in walking mammals to further our understanding of neural encoding and decoding of naturalistic sensory stimuli. However, correlating the intermittent olfactory information with behavior has remained a long-standing challenge due to the stochastic nature of the odor stimulus. We recently reported a method to record real-time olfactory information available to freely moving mice during odor-guided navigation, hence overcoming that challenge. Here we combine our odor-recording method with head-motion tracking to establish correlations between plume encounters and head movements. We show that mice exhibit robust head-pitch motions in the 5-14Hz range during an odor-guided navigation task, and that these head motions are modulated by plume encounters. Furthermore, mice reduce their angles with respect to the source upon plume contact. Head motions may thus be an important part of the sensorimotor behavioral repertoire during naturalistic odor-source localization.

From sight to insight: A multi-task approach with the visual language decoding model

Visual neural decoding aims to unlock the mysteries of how the human brain interprets the visual world through predicting perceived visual information from visual neural activity. While early studies made some progress in decoding visual activity for singular type of information, they failed to concurrently reveal the multi-level interweaving linguistic information in the brain. Here, we developed a novel Visual Language Decoding Model (VLDM) that can simultaneously decode the main categories, semantic labels, and textual descriptions of visual stimuli from visual activities. The large-scale NSD dataset was utilized to ensure the efficiency of the decoding model in joint training and evaluation across multiple tasks. For category decoding, we achieved the effective classification of 12 categories with an accuracy of nearly 70 %, significantly surpassing the chance level. For label decoding, we attained the precise prediction of 80 specific semantic labels with a 16-fold improvement over the chance level. For text decoding, the scores of the decoded text surpassed the corresponding baseline levels by remarkable margins on six evaluation metrics. These results highlight the complexity of how the brain processes visual information and the close connection between visual perception and language cognition. This study contributes significantly to extensive applications in multi-layered brain-computer interfaces, potentially leading to more natural and efficient human-computer interaction experiences.

From long-term to short-term: Distinct neural networks underlying semantic knowledge and its recruitment in working memory

Although numerous studies suggest that working memory (WM) and semantic long-term knowledge interact, the nature and underlying neural mechanisms of this intervention remain poorly understood. Using functional magnetic resonance imaging (fMRI), this study investigated the extent to which neural markers of semantic knowledge in long-term memory (LTM) are activated during the WM maintenance stage in 32 young adults. First, the multivariate neural patterns associated with four semantic categories were determined via an implicit semantic activation task. Next, the participants maintained words – the names of the four semantic categories implicitly activated in the first task – in a verbal WM task. Multi-voxel pattern analyses showed reliable neural decoding of the four semantic categories in the implicit semantic activation and the verbal WM tasks. Critically, however, no between-task classification of semantic categories was observed. Searchlight analyses showed that for the WM task, semantic category information could be decoded in anterior temporal areas associated with abstract semantic category knowledge. In the implicit semantic activation task, semantic category information was decoded in superior temporal, occipital and frontal cortices associated with domain-specific semantic feature representations. These results indicate that item-level semantic activation during verbal WM involves shallow rather than deep semantic information.

Traffic prediction based on spatial-temporal disentangled generative models

Understanding the underlying spatial-temporal patterns in a traffic network is crucial for accurate predictions. However, current deep learning techniques often struggle to effectively identify the generative factors hidden in spatial-temporal traffic data, leading to a lack of interpretability. To address this challenge, we have developed an innovative generative model called Spatial-Temporal Disentangled Neural Relational Inference (STDNRI) for traffic prediction and disentangling interpretable generative factors within observed traffic network data. In this study, we assume that spatial-temporal data is dominated by three generative factors: time-correlated factors, spatial-correlated factors and spatial-temporal joint correlation factors. Based on information bottleneck theory, we establish a novel objective function to maximize the disentanglement of different factors. To model the probability distributions of three factors, we propose three graph neural network encoders to represent time-correlated factors using a graph containing only self-loops, and denote spatial-correlated factors as well as spatial-temporal joint correlation factors adopting a fully-connected graph without self-loops. Then, graph neural network decoder with a spatial-temporal message passing mechanism is used to generate the final predictions. Several comprehensive experiments were conducted based on two real datasets from NYC Yellow Taxi and SHMetro. The results indicate that our STDNRI outperforms ten existing competitive models in terms of high prediction accuracy and provides excellent interpretability for generative factors. Furthermore, the disentangled interpretable factors can afford valuable insights for planners, urban developers, and other stakeholders to gain a deeper understanding of the dynamics of the traffic network, thus enabling more informed decision-making.

Decomposing Task-Relevant Information From Surface Electromyogram for User-Generic Dexterous Finger Force Decoding.

Existing electromyographic (EMG) based motor intent detection algorithms are typically user-specific, and a generic model that can quickly adapt to new users is highly desirable. However, establishing such a model remains a challenge due to high inter-person variability and external interference with EMG signals. In this study, we present a feature disentanglement approach, implemented by an autoencoder-like architecture, designed to decompose user-invariant, motor-task-sensitive high-level representations from user-sensitive, task-irrelevant representations in EMG amplitude features. Our method is user-generic and can be applied to unseen users for continuous multi-finger force predictions. We evaluated our approach on eight subjects, predicting the force of three fingers (index, middle, and ring-pinky) concurrently. We assessed the decoder's performance through a rigorous leave-one-subject-out validation. Our developed approach consistently outperformed both the conventional EMG amplitude method and a commonly used feature projection approach, principal component analysis (PCA), with a lower force prediction error (RMSE: 6.91 ± 0.45 % MVC; R2: 0.835 ± 0.026) and a higher finger classification accuracy (83.0 ± 4.5%). The comparison with the state-of-the-art neural networks further demonstrated the superior performance of our method in user-generic force predictions. Overall, our methods provide novel insights into the development of user-generic and accurate neural decoding for myoelectric control of assistive robotic hands.