Brain-like Neural Network Research Articles

Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse

Deep artificial neural networks (ANNs) play a major role in modeling the visual pathways of primate and rodent. However, they highly simplify the computational properties of neurons compared to their biological counterparts. Instead, Spiking Neural Networks (SNNs) are more biologically plausible models since spiking neurons encode information with time sequences of spikes, just like biological neurons do. However, there is a lack of studies on visual pathways with deep SNNs models. In this study, we model the visual cortex with deep SNNs for the first time, and also with a wide range of state-of-the-art deep CNNs and ViTs for comparison. Using three similarity metrics, we conduct neural representation similarity experiments on three neural datasets collected from two species under three types of stimuli. Based on extensive similarity analyses, we further investigate the functional hierarchy and mechanisms across species. Almost all similarity scores of SNNs are higher than their counterparts of CNNs with an average of 6.6%. Depths of the layers with the highest similarity scores exhibit little differences across mouse cortical regions, but vary significantly across macaque regions, suggesting that the visual processing structure of mice is more regionally homogeneous than that of macaques. Besides, the multi-branch structures observed in some top mouse brain-like neural networks provide computational evidence of parallel processing streams in mice, and the different performance in fitting macaque neural representations under different stimuli exhibits the functional specialization of information processing in macaques. Taken together, our study demonstrates that SNNs could serve as promising candidates to better model and explain the functional hierarchy and mechanisms of the visual system.

A Survey on Brain Effective Connectivity Network Learning.

Human brain effective connectivity characterizes the causal effects of neural activities among different brain regions. Studies of brain effective connectivity networks (ECNs) for different populations contribute significantly to the understanding of the pathological mechanism associated with neuropsychiatric diseases and facilitate finding new brain network imaging markers for the early diagnosis and evaluation for the treatment of cerebral diseases. A deeper understanding of brain ECNs also greatly promotes brain-inspired artificial intelligence (AI) research in the context of brain-like neural networks and machine learning. Thus, how to picture and grasp deeper features of brain ECNs from functional magnetic resonance imaging (fMRI) data is currently an important and active research area of the human brain connectome. In this survey, we first show some typical applications and analyze existing challenging problems in learning brain ECNs from fMRI data. Second, we give a taxonomy of ECN learning methods from the perspective of computational science and describe some representative methods in each category. Third, we summarize commonly used evaluation metrics and conduct a performance comparison of several typical algorithms both on simulated and real datasets. Finally, we present the prospects and references for researchers engaged in learning ECNs.

Multimodal Mood Consistency and Mood Dependency Neural Network Circuit Based on Memristors

The factors that affect learning efficiency in different environments have been considered in many studies, but multimode mood-consistent learning has not been considered specifically. In this paper, a neural network circuit based on memristors to determine multimode mood consistency and mood dependency was constructed. The circuit is composed of a voltage control module, an emotion module, and a synaptic neuron module. Through the voltage control module and emotion module, learning materials with different properties are input into the synaptic neurons. The learning efficiency of different learning materials under different emotions was analyzed in detail. A dual-channel mood-consistent learning was realized, and the mood dependency was further considered. The influence of different channels on the learning was studied to provide ideas for the future development of intelligent brain-like neural networks.

A Novel Method for Constructing EEG Large-Scale Cortical Dynamical Functional Network Connectivity (dFNC): WTCS.

As a kind of biological network, the brain network conduces to understanding the mystery of high-efficiency information processing in the brain, which will provide instructions to develop efficient brain-like neural networks. Large-scale dynamical functional network connectivity (dFNC) provides a more context-sensitive, dynamical, and straightforward sight at a higher network level. Nevertheless, dFNC analysis needs good enough resolution in both temporal and spatial domains, and the construction of dFNC needs to capture the time-varying correlations between two multivariate time series with unmatched spatial dimensions. Effective methods still lack. With well-developed source imaging techniques, electroencephalogram (EEG) has the potential to possess both high temporal and spatial resolutions. Therefore, we proposed to construct the EEG large-scale cortical dFNC based on brain atlas to probe the subtle dynamic activities in the brain and developed a novel method, that is, wavelet coherence-S estimator (WTCS), to assess the dynamic couplings among functional subnetworks with different spatial dimensions. The simulation study demonstrated its robustness and availability of applying to dFNC. The application in real EEG data revealed the appealing "Primary peak" and "P3-like peak" in dFNC network properties and meaningful evolutions in dFNC network topology for P300. Our study brings new insights for probing brain activities at a more dynamical and higher hierarchical level and pushing forward the development of brain-inspired artificial neural networks. The proposed WTCS not only benefits the dFNC studies but also gives a new solution to capture the time-varying couplings between the multivariate time series that is often encountered in signal processing disciplines.

A data-based large-scale model for primary visual cortex enables brain-like robust and versatile visual processing.

We analyze visual processing capabilities of a large-scale model for area V1 that arguably provides the most comprehensive accumulation of anatomical and neurophysiological data to date. We find that this brain-like neural network model can reproduce a number of characteristic visual processing capabilities of the brain, in particular the capability to solve diverse visual processing tasks, also on temporally dispersed visual information, with remarkable robustness to noise. This V1 model, whose architecture and neurons markedly differ from those of deep neural networks used in current artificial intelligence (AI), such as convolutional neural networks (CNNs), also reproduces a number of characteristic neural coding properties of the brain, which provides explanations for its superior noise robustness. Because visual processing is substantially more energy efficient in the brain compared with CNNs in AI, such brain-like neural networks are likely to have an impact on future technology: as blueprints for visual processing in more energy-efficient neuromorphic hardware.

A High-Accuracy and Energy-Efficient CORDIC Based Izhikevich Neuron With Error Suppression and Compensation.

Bio-inspired neuron models are the key building blocks of brain-like neural networks for brain-science exploration and neuromorphic engineering applications. The efficient hardware design of bio-inspired neuron models is one of the challenges to implement brain-like neural networks, as the balancing of model accuracy, energy consumption and hardware cost is very challenging. This paper proposes a high-accuracy and energy-efficient Fast-Convergence COordinate Rotation DIgital Computer (FC-CORDIC) based Izhikevich neuron design. For ensuring the model accuracy, an error propagation model of the Izhikevich neuron is presented for systematic error analysis and effective error reduction. Parameter-Tuning Error Compensation (PTEC) method and Bitwidth-Extension Error Suppression (BEES) method are proposed to reduce the error of Izhikevich neuron design effectively. In addition, by utilizing the FC-CORDIC instead of conventional CORDIC for square calculation in the Izhikevich model, the redundant CORDIC iterations are removed and therefore, both the accumulated errors and required computation are effectively reduced, which significantly improve the accuracy and energy efficiency. An optimized fixed-point design of FC-CORDIC is also proposed to save hardware overhead while ensuring the accuracy. FPGA implementation results exhibit that the proposed Izhikevich neuron design can achieve high accuracy and energy efficiency with an acceptable hardware overhead, among the state-of-the-art designs.

Multiscale Brain-Like Neural Network for Saliency Prediction on Omnidirectional Images

Current top-performing saliency prediction methods of omnidirectional images (ODIs) depend on deep feedforward convolutional neural networks (CNNs), benefiting from their powerful multiscale representation ability. Although these methods adopt deep feedforward CNNs to achieve superb performance in saliency prediction task, they have the following limitations: 1) these deep feedforward CNNs are difficult to map to ventral stream structure of the brain visual system due to their vast number of layers and missing biologically important connections, such as recurrence and 2) most deep feedforward CNNs represent the multiscale features in a layerwise manner. To tackle these issues, models that could learn multiscale features yet share the similarities with human brain are needed. In this article, we propose a novel multiscale brain-like network (MBN) model to predict saliency of head fixations on ODIs. Specifically, our proposed model consists of two major modules: 1) a brain-like CORnet-S module and 2) a multiscale feature extraction module. The CORnet-S module is a lightweight backbone network with four anatomically mapped areas (V1, V2, V4, and IT) and it can simulate the visual processing mechanism of ventral visual stream in the human brain. The multiscale feature extraction module is inspired by the multiscale brain structure, which represents multiscale features at a granular level and increases the range of receptive fields for each network layer. Extensive experiments and ablation studies conducted on two major benchmarks demonstrate the superiority of the proposed MBN model over the state-of-the-art methods.

Brain-Inspired Learning on Neuromorphic Substrates

Neuromorphic hardware strives to emulate brain-like neural networks and thus holds the promise for scalable, low-power information processing on temporal data streams. Yet, to solve real-world problems, these networks need to be trained. However, training on neuromorphic substrates creates significant challenges due to the offline character and the required nonlocal computations of gradient-based learning algorithms. This article provides a mathematical framework for the design of practical online learning algorithms for neuromorphic substrates. Specifically, we show a direct connection between real-time recurrent learning (RTRL), an online algorithm for computing gradients in conventional recurrent neural networks (RNNs), and biologically plausible learning rules for training spiking neural networks (SNNs). Furthermore, we motivate a sparse approximation based on block-diagonal Jacobians, which reduces the algorithm's computational complexity, diminishes the nonlocal information requirements, and empirically leads to good learning performance, thereby improving its applicability to neuromorphic substrates. In summary, our framework bridges the gap between synaptic plasticity and gradient-based approaches from deep learning and lays the foundations for powerful information processing on future neuromorphic hardware systems.

Computing brains: learning algorithms and neurocomputation in the smart city

ABSTRACTThis article examines IBM’s ‘Smarter Education’ program, part of its wider ‘Smarter Cities’ agenda, focusing specifically on its learning analytics applications (based on machine learning algorithms) and cognitive computing developments for education (which take inspiration from neuroscience for the design of brain-like neural networks algorithms and neurocomputational devices). The article conceptualizes the relationship between learning algorithms, neuroscience, and the new learning spaces of the city by combining the notion of programmable ‘code/space’ with ideas about the ‘social life of the brain’ to suggest that new kinds of ‘brain/code/spaces’ are being developed where the environment itself is imagined to possess brain-like functions of learning and ‘human qualities’ of cognition performed by algorithmic processes. IBM’s ambitions for education constitute a sociotechnical imaginary of a ‘cognitive classroom’ where the practices associated with data analytics and cognitive computing in the smart city are being translated into the neuropedagogic brain/code/spaces of the school, with significant consequences for how learners are to be addressed and acted upon. The IBM imaginary of Smarter Education is one significant instantiation of emerging smart cities that are to be governed by neurocomputational processes modelled on neuroscientific insights into the brain’s plasticity for learning, and part of a ‘neurofuture’ in-the-making where nonconscious algorithmic ‘computing brains’ embedded in urban space are intended to interact with human cognition and brain functioning.

Learning Robotic Hand-eye Coordination Through a Developmental Constraint Driven Approach

The skill of robotic hand-eye coordination not only helps robots to deal with real time environment, but also affects the fundamental framework of robotic cognition. A number of approaches have been developed in the literature for construction of the robotic hand-eye coordination. However, several important features within infant developmental procedure have not been introduced into such approaches. This paper proposes a new method for robotic hand-eye coordination by imitating the developmental progress of human infants. The work employs a brain-like neural network system inspired by infant brain structure to learn hand-eye coordination, and adopts a developmental mechanism from psychology to drive the robot. The entire learning procedure is driven by developmental constraint: The robot starts to act under fully constrained conditions, when the robot learning system becomes stable, a new constraint is assigned to the robot. After that, the robot needs to act with this new condition again. When all the contained conditions have been overcome, the robot is able to obtain hand-eye coordination ability. The work is supported by experimental evaluation, which shows that the new approach is able to drive the robot to learn autonomously, and make the robot also exhibit developmental progress similar to human infants.