Brain-inspired Neural Networks Research Articles

Multi-scale full spike pattern for semantic segmentation

Spiking neural networks (SNNs), as the brain-inspired neural networks, encode information in spatio-temporal dynamics. They have the potential to serve as low-power alternatives to artificial neural networks (ANNs) due to their sparse and event-driven nature. However, existing SNN-based models for pixel-level semantic segmentation tasks suffer from poor performance and high memory overhead, failing to fully exploit the computational effectiveness and efficiency of SNNs. To address these challenges, we propose the multi-scale and full spike segmentation network (MFS-Seg), which is based on the deep direct trained SNN and represents the first attempt to train a deep SNN with surrogate gradients for semantic segmentation. Specifically, we design an efficient fully-spike residual block (EFS-Res) to alleviate representation issues caused by spiking noise on different channels. EFS-Res utilizes depthwise separable convolution to improve the distributions of spiking feature maps. The visualization shows that our model can effectively extract the edge features of segmented objects. Furthermore, it can significantly reduce the memory overhead and energy consumption of the network. In addition, we theoretically analyze and prove that EFS-Res can avoid the degradation problem based on block dynamical isometry theory. Experimental results on the Camvid dataset, the DDD17 dataset, and the DSEC-Semantic dataset show that our model achieves comparable performance to the mainstream UNet network with up to 31× fewer parameters, while significantly reducing power consumption by over 13×. Overall, our MFS-Seg model demonstrates promising results in terms of performance, memory efficiency, and energy consumption, showcasing the potential of deep SNNs for semantic segmentation tasks. Our code is available in https://github.com/BICLab/MFS-Seg.

Brain-inspired spiking neural networks in Engineering Mechanics: a new physics-based self-learning framework for sustainable Finite Element analysis

AbstractThe present study aims to develop a sustainable framework employing brain-inspired neural networks for solving boundary value problems in Engineering Mechanics. Spiking neural networks, known as the third generation of artificial neural networks, are proposed for physics-based artificial intelligence. Accompanied by a new pseudo-explicit integration scheme based on spiking recurrent neural networks leading to a spike-based pseudo explicit integration scheme, the underlying differential equations are solved with a physics-informed strategy. We propose additionally a third-generation spike-based Legendre Memory Unit that handles large sequences. These third-generation networks can be implemented on the coming-of-age neuromorphic hardware resulting in less energy and memory consumption. The proposed framework, although implicit, is viewed as a pseudo-explicit scheme since it requires almost no or fewer online training steps to achieve a converged solution even for unseen loading sequences. The proposed framework is deployed in a Finite Element solver for plate structures undergoing cyclic loading and a Xylo-Av2 SynSense neuromorphic chip is used to assess its energy performance. An acceleration of more than 40% when compared to classical Finite Element Method simulations and the capability of online training is observed. We also see a reduction in energy consumption down to the thousandth order.

An FPGA implementation of Bayesian inference with spiking neural networks.

Spiking neural networks (SNNs), as brain-inspired neural network models based on spikes, have the advantage of processing information with low complexity and efficient energy consumption. Currently, there is a growing trend to design hardware accelerators for dedicated SNNs to overcome the limitation of running under the traditional von Neumann architecture. Probabilistic sampling is an effective modeling approach for implementing SNNs to simulate the brain to achieve Bayesian inference. However, sampling consumes considerable time. It is highly demanding for specific hardware implementation of SNN sampling models to accelerate inference operations. Hereby, we design a hardware accelerator based on FPGA to speed up the execution of SNN algorithms by parallelization. We use streaming pipelining and array partitioning operations to achieve model operation acceleration with the least possible resource consumption, and combine the Python productivity for Zynq (PYNQ) framework to implement the model migration to the FPGA while increasing the speed of model operations. We verify the functionality and performance of the hardware architecture on the Xilinx Zynq ZCU104. The experimental results show that the hardware accelerator of the SNN sampling model proposed can significantly improve the computing speed while ensuring the accuracy of inference. In addition, Bayesian inference for spiking neural networks through the PYNQ framework can fully optimize the high performance and low power consumption of FPGAs in embedded applications. Taken together, our proposed FPGA implementation of Bayesian inference with SNNs has great potential for a wide range of applications, it can be ideal for implementing complex probabilistic model inference in embedded systems.

Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning

We review current and emerging knowledge-informed and brain-inspired cognitive systems for realizing adversarial defenses, eXplainable Artificial Intelligence (XAI), and zero-shot or few-shot learning. Data-driven machine learning models have achieved remarkable performance and demonstrated capabilities surpassing humans in many applications. Yet, their inability to exploit domain knowledge leads to serious performance limitations in practical applications. In particular, deep learning systems are exposed to adversarial attacks, which can trick them into making glaringly incorrect decisions. Moreover, complex data-driven models typically lack interpretability or explainability, i.e., their decisions cannot be understood by human subjects. Furthermore, models are usually trained on standard datasets with a closed-world assumption. Hence, they struggle to generalize to unseen cases during inference in practical open-world environments, thus, raising the zero- or few-shot generalization problem. Although many conventional solutions exist, explicit domain knowledge, brain-inspired neural networks and cognitive architectures offer powerful new dimensions towards alleviating these problems. Prior knowledge is represented in appropriate forms like mathematical relations, logic rules, knowledge graphs, and large language models (LLMs). and incorporated in deep learning frameworks to improve performance. Brain-inspired cognition methods use computational models that mimic the human brain to enhance intelligent behavior in artificial agents and autonomous robots. Ultimately, these models achieve better explainability, higher adversarial robustness and data-efficient learning, and can, in turn, provide insights for cognitive science and neuroscience—that is, to deepen human understanding on how the brain works in general, and how it handles these problems.

Theories and methods for large-scale brain-inspired neural networks

Theories and methods for large-scale brain-inspired neural networks

Spatiotemporal Modeling of Grip Forces Captures Proficiency in Manual Robot Control.

New technologies for monitoring grip forces during hand and finger movements in non-standard task contexts have provided unprecedented functional insights into somatosensory cognition. Somatosensory cognition is the basis of our ability to manipulate and transform objects of the physical world and to grasp them with the right amount of force. In previous work, the wireless tracking of grip-force signals recorded from biosensors in the palm of the human hand has permitted us to unravel some of the functional synergies that underlie perceptual and motor learning under conditions of non-standard and essentially unreliable sensory input. This paper builds on this previous work and discusses further, functionally motivated, analyses of individual grip-force data in manual robot control. Grip forces were recorded from various loci in the dominant and non-dominant hands of individuals with wearable wireless sensor technology. Statistical analyses bring to the fore skill-specific temporal variations in thousands of grip forces of a complete novice and a highly proficient expert in manual robot control. A brain-inspired neural network model that uses the output metric of a self-organizing pap with unsupervised winner-take-all learning was run on the sensor output from both hands of each user. The neural network metric expresses the difference between an input representation and its model representation at any given moment in time and reliably captures the differences between novice and expert performance in terms of grip-force variability.Functionally motivated spatiotemporal analysis of individual average grip forces, computed for time windows of constant size in the output of a restricted amount of task-relevant sensors in the dominant (preferred) hand, reveal finger-specific synergies reflecting robotic task skill. The analyses lead the way towards grip-force monitoring in real time. This will permit tracking task skill evolution in trainees, or identify individual proficiency levels in human robot-interaction, which represents unprecedented challenges for perceptual and motor adaptation in environmental contexts of high sensory uncertainty. Cross-disciplinary insights from systems neuroscience and cognitive behavioral science, and the predictive modeling of operator skills using parsimonious Artificial Intelligence (AI), will contribute towards improving the outcome of new types of surgery, in particular the single-port approaches such as NOTES (Natural Orifice Transluminal Endoscopic Surgery) and SILS (Single-Incision Laparoscopic Surgery).

Multi-Net Fusion: Exploring a Brain-Inspired Neural Network Model for Facial Expression Recognition

Multi-Net Fusion: Exploring a Brain-Inspired Neural Network Model for Facial Expression Recognition

Disrupted visual input unveils the computational details of artificial neural networks for face perception.

Convolutional Neural Network (DCNN), with its great performance, has attracted attention of researchers from many disciplines. The studies of the DCNN and that of biological neural systems have inspired each other reciprocally. The brain-inspired neural networks not only achieve great performance but also serve as a computational model of biological neural systems. Here in this study, we trained and tested several typical DCNNs (AlexNet, VGG11, VGG13, VGG16, DenseNet, MobileNet, and EfficientNet) with a face ethnicity categorization task for experiment 1, and an emotion categorization task for experiment 2. We measured the performance of DCNNs by testing them with original and lossy visual inputs (various kinds of image occlusion) and compared their performance with human participants. Moreover, the class activation map (CAM) method allowed us to visualize the foci of the "attention" of these DCNNs. The results suggested that the VGG13 performed the best: Its performance closely resembled human participants in terms of psychophysics measurements, it utilized similar areas of visual inputs as humans, and it had the most consistent performance with inputs having various kinds of impairments. In general, we examined the processing mechanism of DCNNs using a new paradigm and found that VGG13 might be the most human-like DCNN in this task. This study also highlighted a possible paradigm to study and develop DCNNs using human perception as a benchmark.

Post-silicon nano-electronic device and its application in brain-inspired chips

As information technology is moving toward the era of big data, the traditional Von-Neumann architecture shows limitations in performance. The field of computing has already struggled with the latency and bandwidth required to access memory (“the memory wall”) and energy dissipation (“the power wall”). These challenging issues, such as “the memory bottleneck,” call for significant research investments to develop a new architecture for the next generation of computing systems. Brain-inspired computing is a new computing architecture providing a method of high energy efficiency and high real-time performance for artificial intelligence computing. Brain-inspired neural network system is based on neuron and synapse. The memristive device has been proposed as an artificial synapse for creating neuromorphic computer applications. In this study, post-silicon nano-electronic device and its application in brain-inspired chips are surveyed. First, we introduce the development of neural networks and review the current typical brain-inspired chips, including brain-inspired chips dominated by analog circuit and brain-inspired chips of the full-digital circuit, leading to the design of brain-inspired chips based on post-silicon nano-electronic device. Then, through the analysis of N kinds of post-silicon nano-electronic devices, the research progress of constructing brain-inspired chips using post-silicon nano-electronic device is expounded. Lastly, the future of building brain-inspired chips based on post-silicon nano-electronic device has been prospected.

Event-triggered hybrid impulsive control on lag synchronization of delayed memristor-based bidirectional associative memory neural networks for image hiding

Memristor-based bidirectional associative memory neural networks (MBAMNNs) analysis can help to reveal the dynamic basis of secure communication. Effectively representing the synchronization of MBAMNNs is the principal task of applying brain-inspired neural networks to image hiding. Previous research has typically utilized continuous control methods to represent the synchronization of MBAMNNs, but these are not well coordinated under limited network bandwidth. Besides, the special communication features and uncertainties of MBAMNNs will affect the image encryption/decryption by introducing the concept of chaos synchronization for image hiding, but few studies have explored such elements. To address these two issues, we propose an event-triggered hybrid impulsive scheme on lag synchronization for image hiding. Specifically, we incorporate both time-varying uncertainties and the multi-layer topology structure of MBAMNNs into a designed event-triggered scheme. The frequency of the controller can be automatically updated from two novel triggered conditions, and Zeno-behavior can be avoided effectively. Experimental results demonstrate that our scheme not only outperforms several exciting approaches in synchronization but also can effectively realize the image hiding under low energy consumption.

Encompassing Chaos in Brain-inspired Neural Network Models for Substance Identification and Breast Cancer Detection

Encompassing Chaos in Brain-inspired Neural Network Models for Substance Identification and Breast Cancer Detection

PymoNNto: A Flexible Modular Toolbox for Designing Brain-Inspired Neural Networks.

The Python Modular Neural Network Toolbox (PymoNNto) provides a versatile and adaptable Python-based framework to develop and investigate brain-inspired neural networks. In contrast to other commonly used simulators such as Brian2 and NEST, PymoNNto imposes only minimal restrictions for implementation and execution. The basic structure of PymoNNto consists of one network class with several neuron- and synapse-groups. The behaviour of each group can be flexibly defined by exchangeable modules. The implementation of these modules is up to the user and only limited by Python itself. Behaviours can be implemented in Python, Numpy, Tensorflow, and other libraries to perform computations on CPUs and GPUs. PymoNNto comes with convenient high level behaviour modules, allowing differential equation-based implementations similar to Brian2, and an adaptable modular Graphical User Interface for real-time observation and modification of the simulated network and its parameters.

Spiking neural networks take control.

Brain-inspired neural network architecture overcomes unsolved classical control theory problem for telerobotics.

Memristors Based on an Iridium(III) Complex Containing Viologen for Advanced Synaptic Bionics.

Memristors with nonvolatile memory properties are expected to open the era of neuromorphic computing. However, it remains a huge challenge to develop memristors with high uniformity, high stability, and low power consumption for advanced synaptic bionics. Herein, an electroactive iridium(III) complex Ir-vio was designed and synthesized by incorporating a viologen moiety into its N∧N ligand. Complex Ir-vio showed multiple redox states and high sensitivity to an electrical stimulus. Importantly, two-terminal memristors with Ag/Ir-vio/W structure were successfully fabricated by the solution-processable method, which exhibited multilevel storage characteristics with a low switching threshold voltage of 0.5 V and high ON1/ON2/ON3/OFF current ratio of 105/103/102/1 at a low reading bias of 0.05 V. Moreover, the memristors can mimic synaptic plasticity, indicating that they can act as artificial synapses to construct brain-inspired neural networks. The memristive mechanisms can be ascribed to the interconversion among different charge-transfer and redox states under various electrical stimulus. To the best of our knowledge, this work is the first experimental demonstration of memristors based on iridium(III) complexes, opening a new era for the development of synaptic bionic devices based on organometallic compounds.

Optimizing information processing in brain-inspired neural networks

Optimizing information processing in brain-inspired neural networks

Logic Computing with Stateful Neural Networks of Resistive Switches.

Brain-inspired neural networks can process information with high efficiency, thus providing a powerful tool for pattern recognition and other artificial intelligent tasks. By adopting binary inputs/outputs, neural networks can be used to perform Boolean logic operations, thus potentially surpassing complementary metal-oxide-semiconductor logic in terms of area efficiency, execution time, and computing parallelism. Here, the concept of stateful neural networks consisting of resistive switches, which can perform all logic functions with the same network topology, is introduced. The neural network relies on physical computing according to Ohm's law, Kirchhoff 's law, and the ionic migration within an output switch serving as the highly nonlinear activation function. The input and output are nonvolatile resistance states of the devices, thus enabling stateful and cascadable logic operations. Applied voltages provide the synaptic weights, which enable the convenient reconfiguration of the same circuit to serve various logic functions. The neural network can solve all two-input logic operations with just one step, except for the exclusive-OR (XOR) needing two sequential steps. 1-bit full adder operation is shown to take place with just two steps and five resistive switches, thus highlighting the high efficiencies of space, time, and energy of logic computing with the stateful neural network.

Parallel Brain Simulator: A Multi-scale and Parallel Brain-Inspired Neural Network Modeling and Simulation Platform

The brain is naturally a parallel and distributed system. Reverse engineering a cognitive brain is considered to be a grand challenge. In this paper, we present the parallel brain simulator (PBS), a parallel and distributed platform for modeling the cognitive brain at multiple scales. Inspired by large-scale graph computation, PBS can be considered as a universal parallel execution engine, which is aimed at reducing the complexity of distributed programming and providing an easy to use programmable platform for computational neuroscientists and artificial intelligence researchers for modeling and simulation of large-scale neural networks. As illustrative examples and validations, three brain-inspired neural networks which are built on PBS are introduced, including the 1:1 human hippocampus network, the 1:1 mouse whole-brain network and the CASIA brain simulator built for cognitive robotics. We deploy PBS on both commodity clusters and supercomputers, and a scalable performance is achieved. In addition, we provide evaluations on the scalability and performance of both lumped synapse-based simulation and non-lumped synapse-based simulation with different data-graph distribution methods to show the effectiveness and usability of the PBS platform.