Dynamic Neural Network Research Articles

MP-DGCNN for the semantic segmentation of Chinese ancient building point clouds

Point cloud semantic segmentation is a key step in the scan-to-HBIM process. In order to reduce the information in the process of DGCNN, this paper proposes a Mix Pooling Dynamic Graph Convolutional Neural Network (MP-DGCNN) for the segmentation of ancient architecture point clouds. The proposed MP-DGCNN differs from DGCNN mainly in two aspects: (1) to more comprehensively characterize the local topological structure of points, the edge features are redefined, and distance and neighboring points are added to the original edge features; (2) based on a Multilayer Perceptron (MLP), an internal feature adjustment mechanism is established, and a learnable mix pooling operator is designed by fusing adaptive pooling, max pooling, average pooling, and aggregation pooling, to learn local graph features from the point cloud topology. To verify the proposed algorithm, experiments are conducted on the Qutan Temple point cloud dataset, and the results show that compared with PointNet, PointNet++, DGCNN, GACNet and LDGCNN, the MP-DGCNN segmentation network achieves the highest OA, mIOU and mAcc, reaching 90.19%,65.34% and 79.41%, respectively.

Predictive resilience assessment of road networks based on dynamic multi-granularity graph neural network

Due to the influence of global climate anomalies, abnormal weather conditions such as heavy rainfall have become more frequent in recent years, posing a significant threat to the operation of transportation systems. An effective assessment of the resilience of the transportation system before and during heavy rain can alert the transportation department to take necessary emergency actions. However, existing methods for assessing the rainfall resilience of transportation networks mostly suffer from the following problems: (1) Simulation methods for modeling rainfall impacts lack realism; (2) After-the-fact evaluations of resilience cannot offer advance warning prior to or during a heavy rain event. To address above problems, we present a novel resilience assessment methodology for evaluating the resilience of road networks in real-time during heavy rainfall scenarios. In this methodology, we propose the temporal decomposition-based dynamic multi-granularity graph neural network (TD2MG2NN) for long-term traffic speed forecasting, providing a perspective on the future evolution of traffic states for accurate resilience assessment. In addition, we construct a composite traffic resilience indicator, designed to comprehensively reflect changes in the spatial–temporal resilience of the transportation system during heavy rain. Experimental results on four publicly real datasets indicate that the prediction performance of TD2MG2NN outperforms state-of-the-art models. The assessment results for the transportation road network in California demonstrate that the comprehensive resilience indicator is superior to single functional resilience indicator and the real-time methodology for evaluating resilience can accurately depict and predict the operation of the road network system under heavy rainfall scenarios.

Empirical Study on Real Estate Mass Appraisal Based on Dynamic Neural Networks

Real estate mass appraisal is increasingly gaining popularity as a critical issue, reflecting its growing importance and widespread adoption in economic spheres. And data-driven machine learning methods have made new contributions to enhancing the accuracy and intelligence level of mass appraisal. This study employs python web scraping technology to collect raw data on second-hand house transactions spanning from January 2015 to June 2023 in China. Through a series of data processing procedures, including feature indicator acquisition, the removal of irrelevant sample cases, feature indicator quantification, the handling of missing and outlier values, and normalization, a dataset suitable for direct use by mass appraisal models is constructed. A dynamic neural network model composed of three cascaded sub-models is designed, and the optimal parameter combination for model training is identified using grid searching. The appraisal results demonstrate the reliability of the dynamic neural network model proposed in this study, which is applicable to real estate mass appraisal. A comparison with the common methods indicates that the proposed model exhibits a superior performance in real estate mass appraisal.

An exact mathematical description of computation with transient spatiotemporal dynamics in a complex-valued neural network

Networks throughout physics and biology leverage spatiotemporal dynamics for computation. However, the connection between structure and computation remains unclear. Here, we study a complex-valued neural network (cv-NN) with linear interactions and phase-delays. We report the cv-NN displays sophisticated spatiotemporal dynamics, which we then use, in combination with a nonlinear readout, for computation. The cv-NN can instantiate dynamics-based logic gates, encode short-term memories, and mediate secure message passing through a combination of interactions and phase-delays. The computations in this system can be fully described in an exact, closed-form mathematical expression. Finally, using direct intracellular recordings of neurons in slices from neocortex, we demonstrate that computations in the cv-NN are decodable by living biological neurons as the nonlinear readout. These results demonstrate that complex-valued linear systems can perform sophisticated computations, while also being exactly solvable. Taken together, these results open future avenues for design of highly adaptable, bio-hybrid computing systems that can interface seamlessly with other neural networks.

MTdyg: Multi-scale transformers with continuous time dynamic graph model for link prediction

Link prediction through continuous dynamic graph neural networks is a challenging endeavour. Previous studies have considered historic interaction sequences among pairs of nodes. However, this approach does not sufficiently model the links between them. Thus, the frequency of historical common neighbours was proposed to obtain more information about node pairs. Nonetheless, this measure fails to consider the timing of interactions because the relative importance of different interactions shifts over time.Multi-scale transformers with a continuous time dynamic graph model (MTdyg) are our response to this challenge to enhance link prediction. The novel MTdyg model employs a multi-scale patching strategy that dynamically adjusts based on the interaction frequency, feeding segmented interaction sequences into the transformer. This methodology significantly improves the model’s ability to assimilate historical data. Furthermore, we developed a temporal attention-based historical common neighbour encoding technique to effectively identify linkages between source and target nodes in interaction sequences. Extensive testing on eight public datasets confirmed that MTdyg delivers superior performance on most datasets, demonstrating its effectiveness in capturing the nuances of node interactions over time.

Full-Dimensional Proportional-Derivative Control Technique for Turing Pattern and Bifurcation of Delayed Reaction-Diffusion Bidirectional Ring Neural Networks

Abstract In neural networks, the states of neural networks often exhibit significant spatio-temporal heterogeneity due to the diffusion effect of electrons and differences in the concentration of neurotransmitters. One of the macroscopic reflections of this time-spatial inhomogeneity is Turing pattern. However, most current research in reaction-diffusion neural networks has focused only on one-dimensional location information, and the remaining results considering two-dimensional location information are still limited to the case of two neurons. In this paper, we conduct the dynamic analysis and optimal control of a delayed reaction-diffusion neural network model with bidirectional loop structure. First, several mathematical descriptions are given for the proposed neural network model and the full-dimensional partial differential proportional-derivative (PD) controller is introduced. Second, by analyzing the characteristic equation, the conditions for Hopf bifurcation and Turing instability of the controlled network model are obtained. Furthermore, the amplitude equation of the controlled neural network is obtained based on the multiscale analysis method. Subsequently, we determine the key parameters affecting the formation of Turing pattern depending on the amplitude equation. Finally, multiple sets of computer simulations are carried out to support our theoretical results. It is found that the diffusion coefficients and time delays have significant effects on spatio-temporal dynamics of neural networks. Moreover, after reasonable parameter proportioning, the full-dimensional PD control method can alleviate the spatial heterogeneity caused by diffusion projects and time delays.

DGNN-MN: Dynamic Graph Neural Network via memory regenerate and neighbor propagation

DGNN-MN: Dynamic Graph Neural Network via memory regenerate and neighbor propagation

Spectral-switching analysis reveals real-time neuronal network representations of concurrent spontaneous naturalistic behaviors in human brain.

Despite abundant evidence of functional networks in the human brain, their neuronal underpinnings, and relationships to real-time behavior have been challenging to resolve. Analyzing brain-wide intracranial-EEG recordings with video monitoring, acquired in awake subjects during clinical epilepsy evaluation, we discovered the tendency of each brain region to switch back and forth between 2 distinct power spectral densities (PSDs 2-55Hz). We further recognized that this 'spectral switching' occurs synchronously between distant sites, even between regions with differing baseline PSDs, revealing long-range functional networks that would be obscured in analysis of individual frequency bands. Moreover, the real-time PSD-switching dynamics of specific networks exhibited striking alignment with activities such as conversation and hand movements, revealing a multi-threaded functional network representation of concurrent naturalistic behaviors. Network structures and their relationships to behaviors were stable across days, but were altered during N3 sleep. Our results provide a new framework for understanding real-time, brain-wide neural-network dynamics.

Unveiling the impact of low-frequency electrical stimulation on network synchronization and learning behavior in cultured hippocampal neural networks

Understanding the dynamics of neural networks and their response to external stimuli is crucial for unraveling the mechanisms associated with learning processes. In this study, we hypothesized that electrical stimulation (ES) would lead to significant alterations in the activity patterns of hippocampal neuronal networks and investigated the effects of low-frequency ES on hippocampal neuronal populations using the microelectrode arrays (MEAs). Our findings revealed significant alterations in the activity of hippocampal neuronal networks following low-frequency ES trainings. Post-stimulation, the neural activity exhibited an organized burst firing pattern characterized by increased spike and burst firings, increased synchronization, and enhanced learning behaviors. Analysis of peri-stimulus time histograms (PSTHs) further revealed that low-frequency ES (1Hz) significantly enhanced neural plasticity, thereby facilitating the learning process of cultured neurons, whereas high-frequency ES (>10Hz) impeded this process. Moreover, we observed a substantial increase in correlations and connectivity within neuronal networks following ES trainings. These alterations in network properties indicated enhanced synaptic plasticity and emphasized the positive impact of low-frequency ES on hippocampal neural activities, contributing to the brain's capacity for learning and memory.

Improving deep neural network random initialization through neuronal rewiring

The deep learning literature is continuously updated with new architectures and training techniques. However, weight initialization is overlooked by most recent research, despite some intriguing findings regarding random weights. On the other hand, recent works have been approaching Network Science to understand the structure and dynamics of Artificial Neural Networks (ANNs) after training. Therefore, in this work, we analyze the centrality of neurons in randomly initialized networks. We show that a higher neuronal strength variance may decrease performance, while a lower neuronal strength variance usually improves it. A new method is then proposed to rewire neuronal connections according to a preferential attachment (PA) rule based on their strength, which significantly reduces the strength variance of layers initialized by common methods. In this sense, PA rewiring only reorganizes connections, while preserving the magnitude and distribution of the weights. We show through an extensive statistical analysis on image classification tasks that performance is improved in most cases, both during training and testing, when using both simple and complex architectures and learning schedules. Our results show that, aside from the magnitude, the organization of the weights is also relevant for better initialization of deep ANNs.

Coupling Fault Diagnosis Based on Dynamic Vertex Interpretable Graph Neural Network.

Mechanical equipment is composed of several parts, and the interaction between parts exists throughout the whole life cycle, leading to the widespread phenomenon of fault coupling. The diagnosis of independent faults cannot meet the requirements of the health management of mechanical equipment under actual working conditions. In this paper, the dynamic vertex interpretable graph neural network (DIGNN) is proposed to solve the problem of coupling fault diagnosis, in which dynamic vertices are defined in the data topology. First, in the date preprocessing phase, wavelet transform is utilized to make input features interpretable and reduce the uncertainty of model training. In the fault topology, edge connections are made between nodes according to the fault coupling information, and edge connections are established between dynamic nodes and all other nodes. Second the data topology with dynamic vertices is used in the training phase and in the testing phase, the time series data are only fed into dynamic vertices for classification and analysis, which makes it possible to realize coupling fault diagnosis in an industrial production environment. The features extracted in different layers of DIGNN interpret how the model works. The method proposed in this paper can realize the accurate diagnosis of independent faults in the dataset with an accuracy of 100%, and can effectively judge the coupling mode of coupling faults with a comprehensive accuracy of 88.3%.

Dynamic community detection algorithm based on hyperbolic graph convolution

PurposeCommunity detection is a key factor in analyzing the structural features of complex networks. However, traditional dynamic community detection methods often fail to effectively solve the problems of deep network information loss and computational complexity in hyperbolic space. To address this challenge, a hyperbolic space-based dynamic graph neural network community detection model (HSDCDM) is proposed.Design/methodology/approachHSDCDM first projects the node features into the hyperbolic space and then utilizes the hyperbolic graph convolution module on the Poincaré and Lorentz models to realize feature fusion and information transfer. In addition, the parallel optimized temporal memory module ensures fast and accurate capture of time domain information over extended periods. Finally, the community clustering module divides the community structure by combining the node characteristics of the space domain and the time domain. To evaluate the performance of HSDCDM, experiments are conducted on both artificial and real datasets.FindingsExperimental results on complex networks demonstrate that HSDCDM significantly enhances the quality of community detection in hierarchical networks. It shows an average improvement of 7.29% in NMI and a 9.07% increase in ARI across datasets compared to traditional methods. For complex networks with non-Euclidean geometric structures, the HSDCDM model incorporating hyperbolic geometry can better handle the discontinuity of the metric space, provides a more compact embedding that preserves the data structure, and offers advantages over methods based on Euclidean geometry methods.Originality/valueThis model aggregates the potential information of nodes in space through manifold-preserving distribution mapping and hyperbolic graph topology modules. Moreover, it optimizes the Simple Recurrent Unit (SRU) on the hyperbolic space Lorentz model to effectively extract time series data in hyperbolic space, thereby enhancing computing efficiency by eliminating the reliance on tangent space.

Mean-field solution of the neural dynamics in a Greenberg-Hastings model with excitatory and inhibitory units.

We present a mean-field solution of the dynamics of a Greenberg-Hastings neural network with both excitatory and inhibitory units. We analyze the dynamical phase transitions that appear in the stationary state as the model parameters are varied. Analytical solutions are compared with numerical simulations of the microscopic model defined on a fully connected network. We found that the stationary state of this system exhibits a first-order dynamical phase transition (with the associated hysteresis) when the fraction of inhibitory units f is smaller than some critical value f_{t}≲1/2, even for a finite system. Moreover, any solution for f<1/2 can be mapped to a solution for purely excitatory systems (f=0). In finite systems, when the system is dominated by inhibition (f>f_{t}), the first-order transition is replaced by a pseudocritical one, namely a continuous crossover between regions of low and high activity that resembles the finite size behavior of a continuous phase transition order parameter. However, in the thermodynamic limit (i.e., infinite-system-size limit), we found that f_{t}→1/2 and the activity for the inhibition dominated case (f≥f_{t}) becomes negligible for any value of the parameters, while the first-order transition between low- and high-activity phases for f<f_{t} remains.

Electromagnetic radiation control for nonlinear dynamics of Hopfield neural networks.

Electromagnetic radiation (EMR) affects the dynamical behavior of the nervous system, and appropriate EMR helps to study the dynamic mechanism of the nervous system. This paper uses a sophisticated four-dimensional Hopfield neural network (HNN) model augmented with one or more memristors to simulate the effects of EMR. We focus on the chaotic dynamics of HNN under the influence of EMR. Complex dynamical behaviors are found and transient chaotic phenomena have the same initial value sensitivity, showing how transient chaos is affected by EMR. Multiperiodic phenomena induced by quasi-periodic alternations are found in the dual EMR, as well as the suppression properties of the dual EMR for system chaos. This implies that the dynamical behavior of the HNN system can be controlled by varying the amount of EMR or the number of affected neurons in the HNN. Finally, a strong validation of our proposed model is provided by Multisim and Field Programmable Gate Array(FPGA) hardware.

Prediction of three-dimensional ocean temperature in the South China Sea based on time series gridded data and a dynamic spatiotemporal graph neural network

Prediction of three-dimensional ocean temperature in the South China Sea based on time series gridded data and a dynamic spatiotemporal graph neural network

Exploiting Intrinsic Redundancies in Dynamic Graph Neural Networks for Processing Efficiency

Exploiting Intrinsic Redundancies in Dynamic Graph Neural Networks for Processing Efficiency

Enhanced Wireless Communication Optimization with Neural Networks, Proximal Policy Optimization and Edge Computing for Latency and Energy Efficiency

This research proposes a novel approach for efficient resource allocation in wireless communication systems. It combines dynamic neural networks, Proximal Policy Optimization (PPO), and Edge Computing Orchestrator (ECO) for latency-aware and energy-efficient resource allocation. The proposed system integrates multiple components, including a dynamic neural network, PPO, ECO, and a Mobile Edge Computing (MEC) server. The experimental methodology involves utilizing the NS-3 simulation platform to assess latency and energy efficiency in resource allocation within a wireless communication network, incorporating an ECO, MEC server, and dynamic task scheduling algorithms. It demonstrates a holistic and adaptable approach to resource allocation in dynamic environments, showcasing a notable reduction in latency for devices and tasks. Latency values range from 5 to 20 milliseconds, with corresponding resource utilization percentages varying between 80% and 95%. Additionally, energy-efficient resource allocation demonstrates a commendable reduction in energy consumption, with measured values ranging from 10 to 30 watts, coupled with efficient resource usage percentages ranging from 70% to 85%. These outcomes validate the efficacy of achieving both latency-aware and energy-efficient resource allocation for enhanced wireless communication systems. The proposed system has broad applications in healthcare, smart cities, IoT, real-time analytics, autonomous vehicles, and augmented reality, offering a valuable solution to optimize energy consumption, reduce latency, and enhance system efficiency in these industries.

PointDMIG: a dynamic motion-informed graph neural network for 3D action recognition

PointDMIG: a dynamic motion-informed graph neural network for 3D action recognition

A binary-domain recurrent-like architecture-based dynamic graph neural network

The integration of Dynamic Graph Neural Networks (DGNNs) with Smart Manufacturing is crucial as it enables real-time, adaptive analysis of complex data, leading to enhanced predictive accuracy and operational efficiency in industrial environments. To address the problem of poor combination effect and low prediction accuracy of current dynamic graph neural networks in spatial and temporal domains, and over-smoothing caused by traditional graph neural networks, a dynamic graph prediction method based on spatiotemporal binary-domain recurrent-like architecture is proposed: Binary Domain Graph Neural Network (BDGNN). The proposed model begins by utilizing a modified Graph Convolutional Network (GCN) without an activation function to extract meaningful graph topology information, ensuring non-redundant embeddings. In the temporal domain, Recurrent Neural Network (RNN) and residual systems are employed to facilitate the transfer of dynamic graph node information between learner weights, aiming to mitigate the impact of noise within the graph sequence. In the spatial domain, the AdaBoost (Adaptive Boosting) algorithm is applied to replace the traditional approach of stacking layers in a graph neural network. This allows for the utilization of multiple independent graph learners, enabling the extraction of higher-order neighborhood information and alleviating the issue of over-smoothing. The efficacy of BDGNN is evaluated through a series of experiments, with performance metrics including Mean Average Precision (MAP) and Mean Reciprocal Rank (MRR) for link prediction tasks, as well as metrics for traffic speed regression tasks across diverse test sets. Compared with other models, the better experiments results demonstrate that BDGNN model can not only better integrate the connection between time and space information, but also extract higher-order neighbor information to alleviate the over-smoothing phenomenon of the original GCN.

Sparse dynamic graph learning for district heat load forecasting

Accurate heat load forecasting is crucial for the efficient operation and management of district heating systems. This study introduces a novel Sparse Dynamic Graph Neural Network (SDGNN) framework designed to address the complexities of forecasting heat load in district heating networks. The proposed model represents the district heating network as a dynamic graph, with nodes corresponding to consumers or heat sources and edges denoting temporal dependencies. The SDGNN framework comprises three key components: (1) a sparse graph learning module that identifies the most relevant nodes and edges, (2) a spatio-temporal memory enhancement module that captures both short-term and long-term dependencies, and (3) a temporal fusion module that integrates node representations into a comprehensive global forecast. Evaluated on a real-world district heating dataset from Denmark, the SDGNN model demonstrates superior accuracy and efficiency compared to existing methods. The results indicate that the SDGNN framework effectively captures intricate spatio-temporal patterns in historical heat load data, achieving up to 5.7% improvement in RMSE, 7.4% in MAE, and 5.7% in CVRMSE over baseline models. Additionally, incorporating meteorological factors into the model further enhances its predictive performance. These findings suggest that the SDGNN framework is a robust and scalable solution for district heat load forecasting, with potential applications in other domains involving spatio-temporal graph data.