Dynamic Neural Network Research Articles

Predicting the firing behaviour of neural network through the bifurcation analysis of derivative mean-field model

The mean-field model is an important method for understanding the complex dynamics of the nervous system at different spatial levels and for simulating and theoretically analysing the collective dynamic behaviour of large neural populations. In the work, we construct an improved mean-field system of neural networks coupled with quadratic integrate-and-fire neurons and examine the discharge patterns for networks by analysing such a model, which is a three-dimensional smooth differential system. Bifurcation analysis of the mean-field model is conducted from both theoretical and simulation perspectives, we obtain the bifurcation conditions of some co-dimension-two bifurcations and divide the parameter space into different regimes by simulating two parameters bifurcation diagrams. We find a close correspondence, though with some variance, between the mean-field model and neural network when comparing the firing patterns of the two models in various parameter regimes. In summary, the obtained mean-field description builds the bridge between the parameters of neurons or networks and that of a mean-field system to ensure we can compare them and understand the connections between them. Specifically, the mean-field model can reflect the dynamics of neural networks from a macroscopic perspective, and its bifurcation can predict the behaviour of neural networks to a certain extent and understand the mechanisms behind them, such as bursting dynamics of neural networks.

A real time data-driven dynamic glasius bionic neural network path planning algorithm for polar under-ice feature scanning by “Xinghai 1000” AUV

Using Autonomous Underwater Vehicle (AUV) to survey polar under-ice feature is crucial for studying polar marine environment and global climate change. Compared with seafloor terrain scanning, it faces several challenges, such as uncertain under-ice environment, lack of reliable positioning methods, and stringent constraints on recovery location. Thus, taking “Xinghai 1000” as analysis object, this paper proposes a real time data-driven dynamic glasius bionic neural network path planning algorithm (RDGPP), which consists of an online bi-level path planning module. In global planning module, dynamically hierarchical strategy is used to realize task area layering. On this basis, global coverage path is planned by combining neuron activity value updated which utilize real-time environmental information obtained by FLS sonar with multivariant strategy. Moreover, due to AUV recovery mode limitation, return strategy has been devised in multivariant strategy to ensure AUV automatic return to launch point, thereby reducing energy usage. In local planning module, Flagged bit and GBNN-driven A-star algorithm (FGA) is introduced to minimize deadlock escape time for AUV. Simulation experiments verify RDGPP algorithm advancement in key indicators such as turning times, running time, etc., and effectiveness is verified by “Xinghai 1000” under-ice experiment.

Parallel multi-scale dynamic graph neural network for multivariate time series forecasting

Accurately modeling and predicting multivariate time series (MTS) are crucial in real-world scenarios, where precise predictions aid decision-making. MTS exhibit complex variable relationships and dynamic evolution patterns across various time scales, posing significant modeling challenges. To address these, a parallel multi-scale dynamic graph neural network (PMEDGN) is proposed for parallel modeling of dynamic information across multiple time scales. Spatial-temporal embedding module based on spatial-temporal attention mechanisms and multi-scale dynamic graph generation module are designed. These modules capture implicit spatial-temporal dependencies and self-learn to generate multi-scale dynamic graph structures from MTS without predefined graphs, automatically obtaining the importance of different time scale patterns. Additionally, a global graph convolution module is developed to integrate parallel multi-scale information, enhancing collaboration across time scales for final predictions. Comprehensive experiments on real-world datasets demonstrate the effectiveness and superiority of PMEDGN over state-of-the-art methods, underscoring its potential for practical applications.

A centralized power prediction method for large-scale wind power clusters based on dynamic graph neural network

The uncertainty of the wind process leads to the randomness of its regional propagation, and the spatial correlation between nearby wind farms also shows a dynamic change tendency under the influence of wind direction. To consider the influence of dynamic spatial correlation on wind power prediction modeling, a short-term wind power prediction method based on a dynamic graph neural network is proposed. First, the topology graph of the wind farm cluster was established to represent the correlation of wind farms based on graph theory. Then, a dynamic spatiotemporal graph neural network was constructed to adapt graph topology by node embedding, which can explore the changing characteristics of spatial correlation among wind farms. Finally, we proposed a decoupling error model that can quantify the proportion of errors caused by the modeling process, which can assist in evaluating the performance of predictive models. We conducted experiments using data provided by the wind farm cluster in Inner Mongolia, and the average normalized Root Mean Square Error for the 12 wind farms was 0.1424, which verified the effectiveness of the proposed model.

Barcode activity in a recurrent network model of the hippocampus enables efficient memory binding.

Forming an episodic memory requires binding together disparate elements that co-occur in a single experience. One model of this process is that neurons representing different components of a memory bind to an "index" - a subset of neurons unique to that memory. Evidence for this model has recently been found in chickadees, which use hippocampal memory to store and recall locations of cached food. Chickadee hippocampus produces sparse, high-dimensional patterns ("barcodes") that uniquely specify each caching event. Unexpectedly, the same neurons that participate in barcodes also exhibit conventional place tuning. It is unknown how barcode activity is generated, and what role it plays in memory formation and retrieval. It is also unclear how a memory index (e.g. barcodes) could function in the same neural population that represents memory content (e.g. place). Here, we design a biologically plausible model that generates barcodes and uses them to bind experiential content. Our model generates barcodes from place inputs through the chaotic dynamics of a recurrent neural network and uses Hebbian plasticity to store barcodes as attractor states. The model matches experimental observations that memory indices (barcodes) and content signals (place tuning) are randomly intermixed in the activity of single neurons. We demonstrate that barcodes reduce memory interference between correlated experiences. We also show that place tuning plays a complementary role to barcodes, enabling flexible, contextually-appropriate memory retrieval. Finally, our model is compatible with previous models of the hippocampus as generating a predictive map. Distinct predictive and indexing functions of the network are achieved via an adjustment of global recurrent gain. Our results suggest how the hippocampus may use barcodes to resolve fundamental tensions between memory specificity (pattern separation) and flexible recall (pattern completion) in general memory systems.

Research of Video Stream Intensity Limits in UAV FPV Control in Frame Prediction Mode. Part I: Models and Methods

Currently, unmanned aerial vehicles have found wide application in various spheres of the national economy. FPV control refers to a method of controlling unmanned aerial vehicles in which the video stream from the unmanned aircraft is transmitted to the remote pilot station in real time. Due to failures in the communication network, packets with video stream data may be lost or delayed and be delivered late. One of the ways to compensate for the drop in FPS in case of frame loss or delay at the remote pilot station is to predict intermediate frames of the video stream. This paper presents a scheme for predicting intermediate frames of a video stream and presents the results of experiments to establish a realistically possible increase in FPS in a neural network codec, in which various autoencoders such as VQ-f16 are used for video compression as well as the lossless latent feature space compression algorithm DEFLATE, while the dynamic multiscale voxel flow neural network DMVFN is used for prediction. A regression model was developed to predict the prediction time. The task of analyzing FPS for various configurations of the neural network decoder on the side of the external pilot is formalized.

A Robot Ground Medium Classification Algorithm Based on Feature Fusion and Adaptive Spatio-Temporal Cascade Networks

With technological advancements and scientific progress, mobile robots have found widespread applications across various fields. To enable robots to perform tasks safely and effectively in diverse and unknown environments, this paper proposes a ground medium classification algorithm for robots based on feature fusion and an adaptive spatio-temporal cascade network. Specifically, the original directional features in the dataset are first transformed into quaternion form. Then, spatio-temporal forward and reverse neighbors are identified using KD trees, and their connection strengths are evaluated via a kernel density estimation algorithm to determine the final set of neighbors. Subsequently, based on the connection strengths determined in the previous step, we perform noise reduction on the features using discrete wavelet transform. The noise-reduced features are then weighted and fused to generate a new feature representation.After feature fusion, the Adaptive Dynamic Convolutional Neural Network (ADC) proposed in this paper is cascaded with the Long Short-Term Memory (LSTM) network to further extract hybrid spatio-temporal feature information from the dataset, culminating in the final terrain classification. Experiments on the terrain type classification dataset demonstrate that our method achieves an average accuracy of 97.46% and an AUC of 99.80%, significantly outperforming other commonly used algorithms in the field. Furthermore, the effectiveness of each module in the proposed method is further demonstrated through ablation experiments.

Attention-based dynamic multilayer graph neural networks for loan default prediction

Whereas traditional credit scoring tends to employ only individual borrower- or loan-level predictors, it has been acknowledged for some time that connections between borrowers may result in default risk propagating over a network. In this paper, we present a model for credit risk assessment leveraging a dynamic multilayer network built from a Graph Neural Network and a Recurrent Neural Network, each layer reflecting a different source of network connection. We test our methodology in a behavioural credit scoring context using a dataset provided by U.S. mortgage financier Freddie Mac, in which different types of connections arise from the geographical location of the borrower and their choice of mortgage provider. The proposed model considers both types of connections and the evolution of these connections over time. We enhance the model by using a custom attention mechanism that weights the different time snapshots according to their importance. After testing multiple configurations, a model with GAT, LSTM, and the attention mechanism provides the best results. Empirical results demonstrate that, when it comes to predicting probability of default for the borrowers, our proposed model brings both better results and novel insights for the analysis of the importance of connections and timestamps, compared to traditional methods.

Model and Method for Providing Resilience to Resource-Constrained AI-System.

Artificial intelligence technologies are becoming increasingly prevalent in resource-constrained, safety-critical embedded systems. Numerous methods exist to enhance the resilience of AI systems against disruptive influences. However, when resources are limited, ensuring cost-effective resilience becomes crucial. A promising approach for reducing the resource consumption of AI systems during test-time involves applying the concepts and methods of dynamic neural networks. Nevertheless, the resilience of dynamic neural networks against various disturbances remains underexplored. This paper proposes a model architecture and training method that integrate dynamic neural networks with a focus on resilience. Compared to conventional training methods, the proposed approach yields a 24% increase in the resilience of convolutional networks and a 19.7% increase in the resilience of visual transformers under fault injections. Additionally, it results in a 16.9% increase in the resilience of convolutional network ResNet-110 and a 21.6% increase in the resilience of visual transformer DeiT-S under adversarial attacks, while saving more than 30% of computational resources. Meta-training the neural network model improves resilience to task changes by an average of 22%, while achieving the same level of resource savings.

Fire emergency management of large shopping malls: IoT-based evacuee tracking and dynamic path optimization

As urbanization accelerates and buildings become more complex, fire emergency evacuation has become increasingly challenging. Traditional evacuation plans often struggle with slow response times and suboptimal path planning in real-time dynamic and complex fire scenarios. To address these issues, this study proposes the IoT-based DWM-Evac model for fire emergency evacuation path planning. The model leverages IoT technology by using various sensors placed inside buildings to monitor fire incidents and spread in real-time, collecting critical data such as temperature, smoke concentration, and flame location. It integrates Dynamic Graph Neural Networks (DGNN), Whale Optimization Algorithm (WOA), and Markov Decision Process (MDP) to enhance path efficiency and safety. Experimental results indicate that the DWM-Evac model achieves an average evacuation time of 315 s in a virtual mall environment, 25 s shorter than traditional plans, with an average path length of 255 meters and a path safety score of 0.92, higher than the traditional plan’s 0.88. The application of IoT in fire emergency management not only improves response speed but also optimizes path planning, significantly enhancing personnel safety.

Investigation on the regular and chaotic dynamics of a ring network of five inertial Hopfield neural network: theoretical, analog and microcontroller simulation.

The studies conducted in this contribution are based on the analysis of the dynamics of a homogeneous network of five inertial neurons of the Hopfield type to which a unidirectional ring coupling topology is applied. The coupling is achieved by perturbing the next neuron's amplitude with a signal proportional to the previous one. The system consists of ten coupled ODEs, and the investigations carried out have allowed us to highlight several unusual and rarely related dynamics, hence the importance of emphasizing them. The main analysis tools that have helped in obtaining the results presented are phase portraits, bifurcation diagrams, and the Maximal Lyapunov exponent. In this system, we have observed phenomena such as the coexistence of homogeneous and heterogeneous attractors, period-doubling crisis, parallel branches, and the path leading to hyperchaotic multi-spiral. All attractors are non-hidden as they originate from well-known equilibrium points. The system has 254 equilibrium points, among which only 32 undergo a Hopf bifurcation followed by period-doubling, leading to a merging crisis phenomenon until the final hyperchaotic multi-spiral attractor. For the same parameter values (coupling or dissipation), a maximum of 30 attractors for the coupling coefficient and 32 attractors for dissipation coexist, and illustrated by the phase portraits. Virtual verification using Pspice and practical verification using an Arduino Mega 2580 microcontroller of the model have also been reported. They are in perfect agreement with the behaviors resulting from numerical investigations. The circuit energy and dimensionless energy has been estimated and the scale relation established. The results presented further enrich previous and recent work in the study of the nonlinear dynamics of Hopfield-type neural networks. Additionally, it is important to mention that cyclic coupling typology may be used as an alternative approach in generating multi-spiral signals in Hopfield oscillators.

Tai Chi Practice Buffers Aging Effects in Functional Brain Connectivity.

Tai Chi (TC) practice has been shown to improve both cognitive and physical function in older adults. However, the neural mechanisms underlying the benefits of TC remain unclear. Our primary aims are to explore whether distinct age-related and TC-practice-related relationships can be identified with respect to either temporal or spatial (within/between-network connectivity) differences. This cross-sectional study examined recurrent neural network dynamics, employing an adaptive, data-driven thresholding approach to source-localized resting-state EEG data in order to identify meaningful connections across time-varying graphs, using both temporal and spatial features derived from a hidden Markov model (HMM). Mann-Whitney U tests assessed between-group differences in temporal and spatial features by age and TC practice using either healthy younger adult controls (YACs, n = 15), healthy older adult controls (OACs, n = 15), or Tai Chi older adult practitioners (TCOAs, n = 15). Our results showed that aging is associated with decreased within-network and between-network functional connectivity (FC) across most brain networks. Conversely, TC practice appears to mitigate these age-related declines, showing increased FC within and between networks in older adults who practice TC compared to non-practicing older adults. These findings suggest that TC practice may abate age-related declines in neural network efficiency and stability, highlighting its potential as a non-pharmacological intervention for promoting healthy brain aging. This study furthers the triple-network model, showing that a balancing and reorientation of attention might be engaged not only through higher-order and top-down mechanisms (i.e., FPN/DAN) but also via the coupling of bottom-up, sensory-motor (i.e., SMN/VIN) networks.

Purity Skeleton Dynamic Hypergraph Neural Network

Recently, in the field of Hypergraph Neural Networks (HGNNs), the effectiveness of dynamic hypergraph construction has been validated, which aims to reduce structural noise within the hypergraph through embeddings. However, the existing dynamic construction methods fail to notice the reduction of information contained in the hypergraphs during dynamic updates. This limitation undermines the quality of hypergraphs. Moreover, dynamic hypergraphs are constructed from graphs. Several key nodes play a crucial role in graph, but they are overlooked in hypergraphs. In this paper, we propose a Purity Skeleton Dynamic Hypergraph Neural Network (PS-DHGNN) to address the above issues. Firstly, we leverage purity skeleton method to dynamically construct hypergraphs via the fusion embeddings of features and topology simultaneously. This method effectively reduces structural noise and prevents the loss of information. Secondly, we employ an incremental training strategy, which implements a batch training strategy based on the importance of nodes. The key nodes, as the skeleton of hypergraph, are still highly valued. In addition, we utilize a novel loss function for learning structure information between hypergraph and graph. We conduct extensive experiments on node classification and clustering tasks, which demonstrate that our PS-DHGNN outperforms state-of-the-art methods. Note on real-world traffic flow datasets, PS-DHGNN demonstrates excellent performance, which is highly meaningful in practice.

Dynamics of a two-neuron hopfield neural network: Memristive synapse and autapses and impact of fractional order

There are numerous studies on Hopfield neural networks with electromagnetic induction using memristors in either autaptic or synaptic connections. In this study, we explore a novel scenario where all connections are influenced by electromagnetic induction. We investigate and compare the network’s dynamics with one memristive autapse, two memristive autapses, and a memristive synapse. The results indicate that having two memristive autapses instead of one increases the dynamical range, leading to chaotic dynamics in unequal autaptic strengths. In contrast, in the presence of the memristive synapse, chaos can emerge only in very strong synaptic strength. Using fractional-order derivatives can transform the periodic attractor of the integer-order model into a chaotic one in some parameters. Furthermore, incorporating more memristors leads to chaos at lower fractional orders.

Dynamic Neural Network Predictive Compensation-Based Point-to-Point Iterative Learning Control With Nonuniform Batch Length.

This article discusses the problem of nonuniform running length in incomplete tracking control, which often occurs in industrial processes due to artificial or environmental changes, such as chemical engineering. It affects the design and application of iterative learning control (ILC) that relies on the strictly repetitive property. Therefore, a dynamic neural network (NN) predictive compensation strategy is proposed under the point-to-point ILC framework. To handle the difficulty of establishing an accurate mechanism model for real process control, the data-driven approach is also introduced. First, applying the iterative dynamic linearization (IDL) technique and radial basis function NN (RBFNN) to construct the iterative dynamic predictive data model (IDPDM) relies on input-output (I/O) signal, and the extended variable is defined by a predictive model to compensate for the incomplete operation length. Then, a learning algorithm based on multiple iteration errors is proposed using an objective function. This learning gain is constantly updated through the NN to adapt to changes in the system. In addition, the composite energy function (CEF) and compression mapping prove that the system is convergent. Finally, two numerical simulation examples are given.

Backpropagation-Based Learning Techniques for Deep Spiking Neural Networks: A Survey.

With the adoption of smart systems, artificial neural networks (ANNs) have become ubiquitous. Conventional ANN implementations have high energy consumption, limiting their use in embedded and mobile applications. Spiking neural networks (SNNs) mimic the dynamics of biological neural networks by distributing information over time through binary spikes. Neuromorphic hardware has emerged to leverage the characteristics of SNNs, such as asynchronous processing and high activation sparsity. Therefore, SNNs have recently gained interest in the machine learning community as a brain-inspired alternative to ANNs for low-power applications. However, the discrete representation of the information makes the training of SNNs by backpropagation-based techniques challenging. In this survey, we review training strategies for deep SNNs targeting deep learning applications such as image processing. We start with methods based on the conversion from an ANN to an SNN and compare these with backpropagation-based techniques. We propose a new taxonomy of spiking backpropagation algorithms into three categories, namely, spatial, spatiotemporal, and single-spike approaches. In addition, we analyze different strategies to improve accuracy, latency, and sparsity, such as regularization methods, training hybridization, and tuning of the parameters specific to the SNN neuron model. We highlight the impact of input encoding, network architecture, and training strategy on the accuracy-latency tradeoff. Finally, in light of the remaining challenges for accurate and efficient SNN solutions, we emphasize the importance of joint hardware-software codevelopment.

Task-specific topology of brain networks supporting working memory and inhibition.

Network neuroscience explores the brain's connectome, demonstrating that dynamic neural networks support cognitive functions. This study investigates how distinct cognitive abilities-working memory and cognitive inhibitory control-are supported by unique brain network configurations constructed by estimating whole-brain networks using mutual information. The study involved 195 participants who completed the Sternberg Item Recognition task and Flanker tasks while undergoing electroencephalography recording. A mixed-effects linear model analyzed the influence of network metrics on cognitive performance, considering individual differences and task-specific dynamics. The findings indicate that working memory and cognitive inhibitory control are associated with different network attributes, with working memory relying on distributed networks and cognitive inhibitory control on more segregated ones. Our analysis suggests that both strong and weak connections contribute to cognitive processes, with weak connections potentially leading to a more stable and support networks of memory and cognitive inhibitory control. The findings indirectly support the network neuroscience theory of intelligence, suggesting different functional topology of networks inherent to various cognitive functions. Nevertheless, we propose that understanding individual variations in cognitive abilities requires recognizing both shared and unique processes within the brain's network dynamics.

Degradation assessment of wind turbine based on additional load measurements

In recent years, there has been an increasing focus on the degradation assessment of wind turbines using supervisory control and data acquisition (SCADA) data, establishing condition monitoring as the preferred approach. However, SCADA data primarily focuses on general condition variables, such as temperature, while overlooking critical metrics like load factors for assessing the mechanical properties of wind turbines. The load parameters can be utilized to assess the mechanical stresses and strains experienced by wind turbine components, thereby offering more precise information on degradation. To address this gap, we explore the feasibility of implementing additional load monitoring for the degradation assessment of wind turbines. Specifically, we design a cost-effective load sensor system for collecting load data from wind turbines. On this basis, a novel degradation assessment method that incorporates additional load data and a nonlinear dynamic state-space neural network model is proposed to extract degradation information from wind turbines. Then, we introduce a dynamic degradation index, derived through trend analysis, which effectively captures and quantifies the degradation levels of wind turbines over time. Finally, a case study using a dataset collected from a real wind farm is provided to validate the proposed method. The results demonstrate a significant enhancement in degradation assessment. By incorporating additional load data, the proposed method exhibits heightened responsiveness to degradation levels and more effectively captures the progression of degradation trends.

Study on the automatic control method for evaporation treatment of wet flue gas desulfurization wastewater in power plants

ABSTRACT DRFNN is used to predict the evaporation state of wastewater, and incremental PID algorithm is combined to adjust the control quantity. By real-time monitoring the state and parameter changes of wastewater evaporation treatment process, online adjusting and optimizing the parameters of DRFNN and incremental PID, the automatic control of desulfurization wastewater evaporation treatment is completed. The experimental results show that this method can achieve more accurate and reliable automatic control, short control time, strong ability to deal with emergencies, and can converge to stability in a short time. This method can effectively reduce the number of early alarms and failures of the system and save maintenance costs. The IAE values of chloride ion (Cl) and mercury (Hg) in this method are 0.3558 and 0.3872, respectively, which proves that the difference between the actual output and the expected output of this method is very small, which proves that the system in this paper has high accuracy.

Structural damage detection and localization via an unsupervised anomaly detection method

This study introduces an unsupervised machine learning framework for damage detection and localization in Structural Health Monitoring (SHM), leveraging dynamic graph convolutional neural networks and Transformer networks. This approach is specifically tailored to overcome the challenge of limited labeled data in SHM, enabling precise analysis and feature synthesis from sensor-derived time series data for accurate damage identification. Incorporating a novel ‘localization score’ enhances the framework’s precision in pinpointing structural damages by integrating data-driven insights with a physics-informed understanding of structural dynamics. Extensive validations on diverse structures, including a benchmark steel structure and a real-world cable-stayed bridge, underscore the framework’s effectiveness in anomaly detection and damage localization, showcasing its potential to safeguard critical infrastructure through advanced data-effective machine learning techniques.