Dynamic Neural Network Research Articles

DGCPPISP: a PPI site prediction model based on dynamic graph convolutional network and two-stage transfer learning

BackgroundProteins play a pivotal role in the diverse array of biological processes, making the precise prediction of protein–protein interaction (PPI) sites critical to numerous disciplines including biology, medicine and pharmacy. While deep learning methods have progressively been implemented for the prediction of PPI sites within proteins, the task of enhancing their predictive performance remains an arduous challenge.ResultsIn this paper, we propose a novel PPI site prediction model (DGCPPISP) based on a dynamic graph convolutional neural network and a two-stage transfer learning strategy. Initially, we implement the transfer learning from dual perspectives, namely feature input and model training that serve to supply efficacious prior knowledge for our model. Subsequently, we construct a network designed for the second stage of training, which is built on the foundation of dynamic graph convolution.ConclusionsTo evaluate its effectiveness, the performance of the DGCPPISP model is scrutinized using two benchmark datasets. The ensuing results demonstrate that DGCPPISP outshines competing methods in terms of performance. Specifically, DGCPPISP surpasses the second-best method, EGRET, by margins of 5.9%, 10.1%, and 13.3% for F1-measure, AUPRC, and MCC metrics respectively on Dset_186_72_PDB164. Similarly, on Dset_331, it eclipses the performance of the runner-up method, HN-PPISP, by 14.5%, 19.8%, and 29.9% respectively.

Enhanced lithium‐ion battery state‐of‐charge estimation for Electric Vehicles using the AOA‐DNN approach

AbstractElectric vehicles (EVs) battery management systems (BMSs) rely on exact state of charge (SoC) estimations to guarantee efficient and safe operation. Lithium‐ion batteries (LIBs) are favored for EVs due to their extended lifespan, high energy density, and minimal self‐discharge and high voltage. To address these issues, this research propose a LIB SoC prediction based on an actual BMS in EVs. The main objective is improving SoC of LIB. The proposed hybrid strategy is the combined performance of both the dynamic neural networks (DNN) and arithmetic optimization algorithm (AOA). Commonly it is named as DNN‐AOA technique. The SoC of Lithium‐ion batteries are predicted using the DNN approach. The proposed AOA is used to optimize the weight parameter of DNN to enhance prediction accuracy and reliability. By then, the operational MATLAB platform has adopted the proposed framework, and existing procedures are used to compute its execution. The proposed method demonstrates superior existing like Bayesian network (DBN), random vector functional link neural network (RVFLNN) and Gaussian progress regression (GPR). The proposed method yields a lower error value of 0.1 and a higher accuracy value of 98% compared with other existing methods.

Understanding stacking fault energy of NbMoTaW multi-principal element alloys by interpretable machine learning

Stacking fault energy (SFE) is a crucial property influencing the deformation mechanisms of multi-principal element alloys (MPEAs). However, experimentally measuring SFE and exploring composition space of MPEAs is challenging. This study explores the SFE in NbMoTaW by integrating machine learning force fields (ML-FF), molecular dynamics simulations (MD), neural networks, and symbolic regression (SR) methods. A SFE dataset containing 2000 different NbMoTaW compositions were generated by combining ML-FF and MD. Then the neural-network model was developed for SFE prediction with an accuracy of 0.981 and a mean absolute error of 0.020. Using SR, the valence electron concentration (VEC), average shear modulus (G), radii gamma (γ) were identified as the key descriptors to determine SFE with the relationship of SFE=0.307·VEC+0.352·(G+γ). This work demonstrated a significant impact of chemical composition on SFE and established an accurate mathematical expression for SFE prediction, enhancing the understanding and alloy design of MPEAs.

Neuromodulatory Compensation of Cortical Neural Activity on Electrodeposited Pt/Ir Modified Microelectrode Arrays for Temperature Transients.

Temperature has a profound influence on various neuromodulation processes and has emerged as a focal point. However, the effects of acute environmental temperature fluctuations on cultured cortical networks have been inadequately elucidated. To bridge this gap, we have developed a brain-on-a-chip platform integrating cortical networks and electrodeposited Pt/Ir modified microelectrode arrays (MEAs) with 3D-printed bear-shaped triple chambers, facilitating control of temperature transients. This innovative system administers thermal stimuli while concurrently monitoring neuronal activity, including spikes and local field potentials, from 60 microelectrodes (diameter: 30 μm; impedance: 9.34 ± 1.37 kΩ; and phase delay: -45.26 ± 2.85°). Temperature transitions of approximately ±10 °C/s were applied to cortical networks on MEAs via in situ perfusion within the triple chambers. Subsequently, we examined the spatiotemporal dynamics of the brain-on-a-chip under temperature regulation at both the group level (neuronal population) and their interactions (network dynamics) and the individual level (cellular activity). Specifically, we found that after the temperature reduction neurons enhanced the overall information transmission efficiency of the network through synchronous firing to compensate for the decreased efficiency of single-cell level information transmission, in contrast to temperature elevation. By leveraging the integration of high-performance MEAs with perfusion chambers, this investigation provides a comprehensive understanding of the impact of temperature on the spatiotemporal dynamics of neural networks, thereby facilitating future exploration of the intricate interplay between temperature and brain function.

A Rotating Machinery Fault Diagnosis Method Based on Dynamic Graph Convolution Network and Hard Threshold Denoising.

With the development of precision sensing instruments and data storage devices, the fusion of multi-sensor data in gearbox fault diagnosis has attracted much attention. However, existing methods have difficulty in capturing the local temporal dependencies of multi-sensor monitoring information, and the inescapable noise severely decreases the accuracy of multi-sensor information fusion diagnosis. To address these issues, this paper proposes a fault diagnosis method based on dynamic graph convolutional neural networks and hard threshold denoising. Firstly, considering that the relationships between monitoring data from different sensors change over time, a dynamic graph structure is adopted to model the temporal dependencies of multi-sensor data, and, further, a graph convolutional neural network is constructed to achieve the interaction and feature extraction of temporal information from multi-sensor data. Secondly, to avoid the influence of noise in practical engineering, a hard threshold denoising strategy is designed, and a learnable hard threshold denoising layer is embedded into the graph neural network. Experimental fault datasets from two typical gearbox fault test benches under environmental noise are used to verify the effectiveness of the proposed method in gearbox fault diagnosis. The experimental results show that the proposed DDGCN method achieves an average diagnostic accuracy of up to 99.7% under different levels of environmental noise, demonstrating good noise resistance.

Carbon nanotubes as a basis of metamaterials and nanostructures: Crafting via design optimization

Nanotruss structures made of carbon nanotubes are investigated in two conceptual applications: either as building blocks of metamaterials or for nanostructural applications. The nanotrusses are optimized for different purposes, including various loadings, boundary conditions, parameterizations, objectives and constraints used to formulate optimization problems. The procedure relies on a recently developed framework consisting of molecular dynamics simulations, neural networks and finite elements. This framework is now used in the design optimization of nanostructures and the performances of different popular heuristic optimization methods are compared. Five applications of nanotrusses made of carbon nanotubes are analyzed in detail to investigate the mechanical behavior of such structures and the efficiency of the optimizations. Besides an introductory example, the design of an energy trapping carbon nanotube nanotruss, an auxetic nanotruss, a cantilever nanotruss and the maximization of the compressive strength of a metamaterial are presented. It is shown that the exceptional mechanical properties of carbon nanotubes can indeed be exploited for the development of structures and materials with extraordinary mechanical properties. Although hampered by material and geometrical nonlinearity of the problem, most of the tested optimization methods have proven to be a good choice for the design of such materials and structures.

MP-DGCNN for the semantic segmentation of Chinese ancient building point clouds

Point cloud semantic segmentation is a key step in the scan-to-HBIM process. In order to reduce the information in the process of DGCNN, this paper proposes a Mix Pooling Dynamic Graph Convolutional Neural Network (MP-DGCNN) for the segmentation of ancient architecture point clouds. The proposed MP-DGCNN differs from DGCNN mainly in two aspects: (1) to more comprehensively characterize the local topological structure of points, the edge features are redefined, and distance and neighboring points are added to the original edge features; (2) based on a Multilayer Perceptron (MLP), an internal feature adjustment mechanism is established, and a learnable mix pooling operator is designed by fusing adaptive pooling, max pooling, average pooling, and aggregation pooling, to learn local graph features from the point cloud topology. To verify the proposed algorithm, experiments are conducted on the Qutan Temple point cloud dataset, and the results show that compared with PointNet, PointNet++, DGCNN, GACNet and LDGCNN, the MP-DGCNN segmentation network achieves the highest OA, mIOU and mAcc, reaching 90.19%,65.34% and 79.41%, respectively.

Predictive resilience assessment of road networks based on dynamic multi-granularity graph neural network

Due to the influence of global climate anomalies, abnormal weather conditions such as heavy rainfall have become more frequent in recent years, posing a significant threat to the operation of transportation systems. An effective assessment of the resilience of the transportation system before and during heavy rain can alert the transportation department to take necessary emergency actions. However, existing methods for assessing the rainfall resilience of transportation networks mostly suffer from the following problems: (1) Simulation methods for modeling rainfall impacts lack realism; (2) After-the-fact evaluations of resilience cannot offer advance warning prior to or during a heavy rain event. To address above problems, we present a novel resilience assessment methodology for evaluating the resilience of road networks in real-time during heavy rainfall scenarios. In this methodology, we propose the temporal decomposition-based dynamic multi-granularity graph neural network (TD2MG2NN) for long-term traffic speed forecasting, providing a perspective on the future evolution of traffic states for accurate resilience assessment. In addition, we construct a composite traffic resilience indicator, designed to comprehensively reflect changes in the spatial–temporal resilience of the transportation system during heavy rain. Experimental results on four publicly real datasets indicate that the prediction performance of TD2MG2NN outperforms state-of-the-art models. The assessment results for the transportation road network in California demonstrate that the comprehensive resilience indicator is superior to single functional resilience indicator and the real-time methodology for evaluating resilience can accurately depict and predict the operation of the road network system under heavy rainfall scenarios.

Empirical Study on Real Estate Mass Appraisal Based on Dynamic Neural Networks

Real estate mass appraisal is increasingly gaining popularity as a critical issue, reflecting its growing importance and widespread adoption in economic spheres. And data-driven machine learning methods have made new contributions to enhancing the accuracy and intelligence level of mass appraisal. This study employs python web scraping technology to collect raw data on second-hand house transactions spanning from January 2015 to June 2023 in China. Through a series of data processing procedures, including feature indicator acquisition, the removal of irrelevant sample cases, feature indicator quantification, the handling of missing and outlier values, and normalization, a dataset suitable for direct use by mass appraisal models is constructed. A dynamic neural network model composed of three cascaded sub-models is designed, and the optimal parameter combination for model training is identified using grid searching. The appraisal results demonstrate the reliability of the dynamic neural network model proposed in this study, which is applicable to real estate mass appraisal. A comparison with the common methods indicates that the proposed model exhibits a superior performance in real estate mass appraisal.

An exact mathematical description of computation with transient spatiotemporal dynamics in a complex-valued neural network

Networks throughout physics and biology leverage spatiotemporal dynamics for computation. However, the connection between structure and computation remains unclear. Here, we study a complex-valued neural network (cv-NN) with linear interactions and phase-delays. We report the cv-NN displays sophisticated spatiotemporal dynamics, which we then use, in combination with a nonlinear readout, for computation. The cv-NN can instantiate dynamics-based logic gates, encode short-term memories, and mediate secure message passing through a combination of interactions and phase-delays. The computations in this system can be fully described in an exact, closed-form mathematical expression. Finally, using direct intracellular recordings of neurons in slices from neocortex, we demonstrate that computations in the cv-NN are decodable by living biological neurons as the nonlinear readout. These results demonstrate that complex-valued linear systems can perform sophisticated computations, while also being exactly solvable. Taken together, these results open future avenues for design of highly adaptable, bio-hybrid computing systems that can interface seamlessly with other neural networks.

MTdyg: Multi-scale transformers with continuous time dynamic graph model for link prediction

Link prediction through continuous dynamic graph neural networks is a challenging endeavour. Previous studies have considered historic interaction sequences among pairs of nodes. However, this approach does not sufficiently model the links between them. Thus, the frequency of historical common neighbours was proposed to obtain more information about node pairs. Nonetheless, this measure fails to consider the timing of interactions because the relative importance of different interactions shifts over time.Multi-scale transformers with a continuous time dynamic graph model (MTdyg) are our response to this challenge to enhance link prediction. The novel MTdyg model employs a multi-scale patching strategy that dynamically adjusts based on the interaction frequency, feeding segmented interaction sequences into the transformer. This methodology significantly improves the model’s ability to assimilate historical data. Furthermore, we developed a temporal attention-based historical common neighbour encoding technique to effectively identify linkages between source and target nodes in interaction sequences. Extensive testing on eight public datasets confirmed that MTdyg delivers superior performance on most datasets, demonstrating its effectiveness in capturing the nuances of node interactions over time.

Full-Dimensional Proportional-Derivative Control Technique for Turing Pattern and Bifurcation of Delayed Reaction-Diffusion Bidirectional Ring Neural Networks

Abstract In neural networks, the states of neural networks often exhibit significant spatio-temporal heterogeneity due to the diffusion effect of electrons and differences in the concentration of neurotransmitters. One of the macroscopic reflections of this time-spatial inhomogeneity is Turing pattern. However, most current research in reaction-diffusion neural networks has focused only on one-dimensional location information, and the remaining results considering two-dimensional location information are still limited to the case of two neurons. In this paper, we conduct the dynamic analysis and optimal control of a delayed reaction-diffusion neural network model with bidirectional loop structure. First, several mathematical descriptions are given for the proposed neural network model and the full-dimensional partial differential proportional-derivative (PD) controller is introduced. Second, by analyzing the characteristic equation, the conditions for Hopf bifurcation and Turing instability of the controlled network model are obtained. Furthermore, the amplitude equation of the controlled neural network is obtained based on the multiscale analysis method. Subsequently, we determine the key parameters affecting the formation of Turing pattern depending on the amplitude equation. Finally, multiple sets of computer simulations are carried out to support our theoretical results. It is found that the diffusion coefficients and time delays have significant effects on spatio-temporal dynamics of neural networks. Moreover, after reasonable parameter proportioning, the full-dimensional PD control method can alleviate the spatial heterogeneity caused by diffusion projects and time delays.

DGNN-MN: Dynamic Graph Neural Network via memory regenerate and neighbor propagation

DGNN-MN: Dynamic Graph Neural Network via memory regenerate and neighbor propagation

Spectral-switching analysis reveals real-time neuronal network representations of concurrent spontaneous naturalistic behaviors in human brain.

Despite abundant evidence of functional networks in the human brain, their neuronal underpinnings, and relationships to real-time behavior have been challenging to resolve. Analyzing brain-wide intracranial-EEG recordings with video monitoring, acquired in awake subjects during clinical epilepsy evaluation, we discovered the tendency of each brain region to switch back and forth between 2 distinct power spectral densities (PSDs 2-55Hz). We further recognized that this 'spectral switching' occurs synchronously between distant sites, even between regions with differing baseline PSDs, revealing long-range functional networks that would be obscured in analysis of individual frequency bands. Moreover, the real-time PSD-switching dynamics of specific networks exhibited striking alignment with activities such as conversation and hand movements, revealing a multi-threaded functional network representation of concurrent naturalistic behaviors. Network structures and their relationships to behaviors were stable across days, but were altered during N3 sleep. Our results provide a new framework for understanding real-time, brain-wide neural-network dynamics.

Unveiling the impact of low-frequency electrical stimulation on network synchronization and learning behavior in cultured hippocampal neural networks

Understanding the dynamics of neural networks and their response to external stimuli is crucial for unraveling the mechanisms associated with learning processes. In this study, we hypothesized that electrical stimulation (ES) would lead to significant alterations in the activity patterns of hippocampal neuronal networks and investigated the effects of low-frequency ES on hippocampal neuronal populations using the microelectrode arrays (MEAs). Our findings revealed significant alterations in the activity of hippocampal neuronal networks following low-frequency ES trainings. Post-stimulation, the neural activity exhibited an organized burst firing pattern characterized by increased spike and burst firings, increased synchronization, and enhanced learning behaviors. Analysis of peri-stimulus time histograms (PSTHs) further revealed that low-frequency ES (1Hz) significantly enhanced neural plasticity, thereby facilitating the learning process of cultured neurons, whereas high-frequency ES (>10Hz) impeded this process. Moreover, we observed a substantial increase in correlations and connectivity within neuronal networks following ES trainings. These alterations in network properties indicated enhanced synaptic plasticity and emphasized the positive impact of low-frequency ES on hippocampal neural activities, contributing to the brain's capacity for learning and memory.

Improving deep neural network random initialization through neuronal rewiring

The deep learning literature is continuously updated with new architectures and training techniques. However, weight initialization is overlooked by most recent research, despite some intriguing findings regarding random weights. On the other hand, recent works have been approaching Network Science to understand the structure and dynamics of Artificial Neural Networks (ANNs) after training. Therefore, in this work, we analyze the centrality of neurons in randomly initialized networks. We show that a higher neuronal strength variance may decrease performance, while a lower neuronal strength variance usually improves it. A new method is then proposed to rewire neuronal connections according to a preferential attachment (PA) rule based on their strength, which significantly reduces the strength variance of layers initialized by common methods. In this sense, PA rewiring only reorganizes connections, while preserving the magnitude and distribution of the weights. We show through an extensive statistical analysis on image classification tasks that performance is improved in most cases, both during training and testing, when using both simple and complex architectures and learning schedules. Our results show that, aside from the magnitude, the organization of the weights is also relevant for better initialization of deep ANNs.

Coupling Fault Diagnosis Based on Dynamic Vertex Interpretable Graph Neural Network.

Mechanical equipment is composed of several parts, and the interaction between parts exists throughout the whole life cycle, leading to the widespread phenomenon of fault coupling. The diagnosis of independent faults cannot meet the requirements of the health management of mechanical equipment under actual working conditions. In this paper, the dynamic vertex interpretable graph neural network (DIGNN) is proposed to solve the problem of coupling fault diagnosis, in which dynamic vertices are defined in the data topology. First, in the date preprocessing phase, wavelet transform is utilized to make input features interpretable and reduce the uncertainty of model training. In the fault topology, edge connections are made between nodes according to the fault coupling information, and edge connections are established between dynamic nodes and all other nodes. Second the data topology with dynamic vertices is used in the training phase and in the testing phase, the time series data are only fed into dynamic vertices for classification and analysis, which makes it possible to realize coupling fault diagnosis in an industrial production environment. The features extracted in different layers of DIGNN interpret how the model works. The method proposed in this paper can realize the accurate diagnosis of independent faults in the dataset with an accuracy of 100%, and can effectively judge the coupling mode of coupling faults with a comprehensive accuracy of 88.3%.

Dynamic community detection algorithm based on hyperbolic graph convolution

PurposeCommunity detection is a key factor in analyzing the structural features of complex networks. However, traditional dynamic community detection methods often fail to effectively solve the problems of deep network information loss and computational complexity in hyperbolic space. To address this challenge, a hyperbolic space-based dynamic graph neural network community detection model (HSDCDM) is proposed.Design/methodology/approachHSDCDM first projects the node features into the hyperbolic space and then utilizes the hyperbolic graph convolution module on the Poincaré and Lorentz models to realize feature fusion and information transfer. In addition, the parallel optimized temporal memory module ensures fast and accurate capture of time domain information over extended periods. Finally, the community clustering module divides the community structure by combining the node characteristics of the space domain and the time domain. To evaluate the performance of HSDCDM, experiments are conducted on both artificial and real datasets.FindingsExperimental results on complex networks demonstrate that HSDCDM significantly enhances the quality of community detection in hierarchical networks. It shows an average improvement of 7.29% in NMI and a 9.07% increase in ARI across datasets compared to traditional methods. For complex networks with non-Euclidean geometric structures, the HSDCDM model incorporating hyperbolic geometry can better handle the discontinuity of the metric space, provides a more compact embedding that preserves the data structure, and offers advantages over methods based on Euclidean geometry methods.Originality/valueThis model aggregates the potential information of nodes in space through manifold-preserving distribution mapping and hyperbolic graph topology modules. Moreover, it optimizes the Simple Recurrent Unit (SRU) on the hyperbolic space Lorentz model to effectively extract time series data in hyperbolic space, thereby enhancing computing efficiency by eliminating the reliance on tangent space.

Mean-field solution of the neural dynamics in a Greenberg-Hastings model with excitatory and inhibitory units.

We present a mean-field solution of the dynamics of a Greenberg-Hastings neural network with both excitatory and inhibitory units. We analyze the dynamical phase transitions that appear in the stationary state as the model parameters are varied. Analytical solutions are compared with numerical simulations of the microscopic model defined on a fully connected network. We found that the stationary state of this system exhibits a first-order dynamical phase transition (with the associated hysteresis) when the fraction of inhibitory units f is smaller than some critical value f_{t}≲1/2, even for a finite system. Moreover, any solution for f<1/2 can be mapped to a solution for purely excitatory systems (f=0). In finite systems, when the system is dominated by inhibition (f>f_{t}), the first-order transition is replaced by a pseudocritical one, namely a continuous crossover between regions of low and high activity that resembles the finite size behavior of a continuous phase transition order parameter. However, in the thermodynamic limit (i.e., infinite-system-size limit), we found that f_{t}→1/2 and the activity for the inhibition dominated case (f≥f_{t}) becomes negligible for any value of the parameters, while the first-order transition between low- and high-activity phases for f<f_{t} remains.

Electromagnetic radiation control for nonlinear dynamics of Hopfield neural networks.

Electromagnetic radiation (EMR) affects the dynamical behavior of the nervous system, and appropriate EMR helps to study the dynamic mechanism of the nervous system. This paper uses a sophisticated four-dimensional Hopfield neural network (HNN) model augmented with one or more memristors to simulate the effects of EMR. We focus on the chaotic dynamics of HNN under the influence of EMR. Complex dynamical behaviors are found and transient chaotic phenomena have the same initial value sensitivity, showing how transient chaos is affected by EMR. Multiperiodic phenomena induced by quasi-periodic alternations are found in the dual EMR, as well as the suppression properties of the dual EMR for system chaos. This implies that the dynamical behavior of the HNN system can be controlled by varying the amount of EMR or the number of affected neurons in the HNN. Finally, a strong validation of our proposed model is provided by Multisim and Field Programmable Gate Array(FPGA) hardware.