Connection Weights Research Articles

A novel hybrid model for species distribution prediction using neural networks and Grey Wolf Optimizer algorithm

Neural networks are frequently employed to model species distribution through backpropagation methods, known as backpropagation neural networks (BPNN). However, the complex structure of BPNN introduces parameter settings challenges, such as the determination of connection weights, which can affect the accuracy of model simulation. In this paper, we integrated the Grey Wolf Optimizer (GWO) algorithm, renowned for its excellent global search capacity and rapid convergence, to enhance the performance of BPNN. Then we obtained a novel hybrid algorithm, the Grey Wolf Optimizer algorithm optimized backpropagation neural networks algorithm (GNNA), designed for predicting species’ potential distribution. We also compared the GNNA with four prevalent species distribution models (SDMs), namely the generalized boosting model (GBM), generalized linear model (GLM), maximum entropy (MaxEnt), and random forest (RF). These models were evaluated using three evaluation metrics: the area under the receiver operating characteristic curve, Cohen’s kappa, and the true skill statistic, across 23 varied species. Additionally, we examined the predictive accuracy concerning spatial distribution. The results showed that the predictive performance of GNNA was significantly improved compared to BPNN, was significantly better than that of GLM and GBM, and was even comparable to that of MaxEnt and RF in predicting species distributions with small sample sizes. Furthermore, the GNNA demonstrates exceptional powers in forecasting the potential non-native distribution of invasive plant species.

We Put the “UN” in FUN: The Mathematical Guide to Saving the World

We Put the “UN” in FUN: The Mathematical Guide to Saving the World

MEMS Gyroscope Temperature Compensation Based on Improved Complete Ensemble Empirical Mode Decomposition and Optimized Extreme Learning Machine.

Herein, we investigate the temperature compensation for a dual-mass MEMS gyroscope. After introducing and simulating the dual-mass MEMS gyroscope's working modes, we propose a hybrid algorithm for temperature compensation relying on improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN), sample entropy, time-frequency peak filtering, non-dominated sorting genetic algorithm-II (NSGA II) and extreme learning machine. Firstly, we use ICEEMDAN to decompose the gyroscope's output signal, and then we use sample entropy to classify the decomposed signals. For noise segments and mixed segments with different levels of noise, we use time-frequency peak filtering with different window lengths to achieve a trade-off between noise removal and signal retention. For the feature segment with temperature drift, we build a compensation model using extreme learning machine. To improve the compensation accuracy, NSGA II is used to optimize extreme learning machine, with the prediction error and the 2-norm of the output-layer connection weight as the optimization objectives. Enormous simulation experiments prove the excellent performance of our proposed scheme, which can achieve trade-offs in signal decomposition, classification, denoising and compensation. The improvement in the compensated gyroscope's output signal is analyzed based on Allen variance; its angle random walk is decreased from 0.531076°/h/√Hz to 6.65894 × 10-3°/h/√Hz and its bias stability is decreased from 32.7364°/h to 0.259247°/h.

Modeling and simulation for prediction of multiple sclerosis progression

In light of extensive work that has created a wide range of techniques for predicting the course of multiple sclerosis (MS) disease, this paper attempts to provide an overview of these approaches and put forth an alternative way to predict the disease progression. For this purpose, the existing methods for estimating and predicting the course of the disease have been categorized into clinical, radiological, biological, and computational or artificial intelligence-based markers. Weighing the weaknesses and strengths of these prognostic groups is a profound method that is yet in need and works directly at the level of diseased connectivity. Therefore, we propose using the computational models in combination with established connectomes as a predictive tool for MS disease trajectories. The fundamental conduction-based Hodgkin-Huxley model emerged as promising from examining these studies. The advantage of the Hodgkin-Huxley model is that certain properties of connectomes, such as neuronal connection weights, spatial distances, and adjustments of signal transmission rates, can be taken into account. It is precisely these properties that are particularly altered in MS and that have strong implications for processing, transmission, and interactions of neuronal signaling patterns.The Hodgkin-Huxley (HH) equations as a point-neuron model are used for signal propagation inside a small network. The objective is to change the conduction parameter of the neuron model, replicate the changes in myelin properties in MS and observe the dynamics of the signal propagation across the network. The model is initially validated for different lengths, conduction values, and connection weights through three nodal connections. Later, these individual factors are incorporated into a small network and simulated to mimic the condition of MS. The signal propagation pattern is observed after inducing changes in conduction parameters at certain nodes in the network and compared against a control model pattern obtained before the changes are applied to the network. The signal propagation pattern varies as expected by adapting to the input conditions. Similarly, when the model is applied to a connectome, the pattern changes could give an insight into disease progression. This approach has opened up a new path to explore the progression of the disease in MS. The work is in its preliminary state, but with a future vision to apply this method in a connectome, providing a better clinical tool.

Optimal reaching subject to computational and physical constraints reveals structure of the sensorimotor control system

Optimal feedback control provides an abstract framework describing the architecture of the sensorimotor system without prescribing implementation details such as what coordinate system to use, how feedback is incorporated, or how to accommodate changing task complexity. We investigate how such details are determined by computational and physical constraints by creating a model of the upper limb sensorimotor system in which all connection weights between neurons, feedback, and muscles are unknown. By optimizing these parameters with respect to an objective function, we find that the model exhibits a preference for an intrinsic (joint angle) coordinate representation of inputs and feedback and learns to calculate a weighted feedforward and feedback error. We further show that complex reaches around obstacles can be achieved by augmenting our model with a path-planner based on via points. The path-planner revealed "avoidance" neurons that encode directions to reach around obstacles and "placement" neurons that make fine-tuned adjustments to via point placement. Our results demonstrate the surprising capability of computationally constrained systems and highlight interesting characteristics of the sensorimotor system.

Observer‐based event‐triggered consensus control of nonlinear cyber‐physical systems under backlash‐like hysteresis and denial‐of‐service attacks

AbstractIn this article, we investigate the problem of event‐triggered distributed consensus control for a class of complex cyber‐physical systems (CPSs), where unknown backlash‐like hysteresis inputs and denial‐of‐service (DoS) attacks are considered in actuators and communication channels, respectively. In the first place, when communication networks are subjected to malicious DoS attacks, the connection weights of network communication topologies experience disruptions. To solve this issue, this work proposes a solution by constructing topological models for communication networks employing switching topologies and utilizing observers to estimate the combined measurement errors during the time interval of DoS attacks. Secondly, to address the repercussions of DoS attacks on control units and to optimize the utilization of communication resources, a distributed control protocol based on an event‐triggered scheme and employing locally deployed estimators is designed to significantly decrease the update frequency of control signals. Meanwhile, utilizing Lyapunov stability theory, the boundedness of all signals in the closed‐loop system is validated, thereby accomplishing the goal of consensus control. At last, the efficacy of the proposed control scheme is demonstrated through two simulation examples.

Exploring non-Euclidean relationships in EEG emotion recognition: A dynamic graph attention network

Deep learning classification models based on electroencephalogram (EEG) emotion recognition have demonstrated considerable proficiency in the categorization of emotional states. However, these models have limitations in their capability to analyze the active states and cooperative relationships among distinct brain regions. This study proposes a dynamic graph attention network (DGAT) for EEG emotion recognition, which learns the features of each channel and leverages multiple-head self-attention mechanisms to capture non-Euclidean relationships between channels. Then, we use differential entropy features of emotions signals on the SJTU emotion EEG dataset (SEED). The DGAT model achieved improved subject-dependent and cross-subject classification accuracy compared to previous models. Moreover, ablation studies show that the channel weight matrix(CWM) and appropriate hyper-parameters can improve the performance of the DGAT model significantly. Furthermore, by conducting interpretable analysis of the new connections and electrode weights learned by the model, we find that these connection weight relationships reflect a certain degree of coordination within the brain for EEG-based emotion recognition. These findings provide a new method for EEG emotion recognition and highlights the potential for using deep learning models to analyze the active state and synergistic relationships among brain regions.

Structural constraints on the emergence of oscillations in multi-population neural networks

Oscillations arise in many real-world systems and are associated with both functional and dysfunctional states. Whether a network can oscillate can be estimated if we know the strength of interaction between nodes. But in real-world networks (in particular in biological networks) it is usually not possible to know the exact connection weights. Therefore, it is important to determine the structural properties of a network necessary to generate oscillations. Here, we provide a proof that uses dynamical system theory to prove that an odd number of inhibitory nodes and strong enough connections are necessary to generate oscillations in a single cycle threshold-linear network. We illustrate these analytical results in a biologically plausible network with either firing-rate based or spiking neurons. Our work provides structural properties necessary to generate oscillations in a network. We use this knowledge to reconcile recent experimental findings about oscillations in basal ganglia with classical findings.

Structural constraints on the emergence of oscillations in multi-population neural networks.

Oscillations arise in many real-world systems and are associated with both functional and dysfunctional states. Whether a network can oscillate can be estimated if we know the strength of interaction between nodes. But in real-world networks (in particular in biological networks) it is usually not possible to know the exact connection weights. Therefore, it is important to determine the structural properties of a network necessary to generate oscillations. Here, we provide a proof that uses dynamical system theory to prove that an odd number of inhibitory nodes and strong enough connections are necessary to generate oscillations in a single cycle threshold-linear network. We illustrate these analytical results in a biologically plausible network with either firing-rate based or spiking neurons. Our work provides structural properties necessary to generate oscillations in a network. We use this knowledge to reconcile recent experimental findings about oscillations in basal ganglia with classical findings.

Prediction intervals for carbon dioxide emissions in china by extreme learning machine ensemble based on particle swarm optimization

<p>The emission of carbon dioxide is the major cause of the greenhouse effect, which has a negative impact on human survival and sustainable economic development; therefore, it is very significant for discovering the underlying influential factors of carbon dioxide emissions. In this paper, an extreme learning machine ensemble based on particle swarm optimization approach(PSO-ELMEnsemble) are applied to predict the emission of carbon dioxide, which provide estimated values as well as the corresponding reliability. In addition, the particle swarm optimization approach is used to optimize the connection weights of extreme learning machine. The performance of the proposed PSO-ELMEnsemble is experimentally validated by carbon dioxide emissions data during the period 1978-2016 by using state-of-the-art comparative method, where the proposed approach outperforms the others in achieving the best trade-off between accuracy and simplicity.</p>

Multipattern graph convolutional network-based autism spectrum disorder identification.

The early diagnosis of autism spectrum disorder (ASD) has been extensively facilitated through the utilization of resting-state fMRI (rs-fMRI). With rs-fMRI, the functional brain network (FBN) has gained much attention in diagnosing ASD. As a promising strategy, graph convolutional networks (GCN) provide an attractive approach to simultaneously extract FBN features and facilitate ASD identification, thus replacing the manual feature extraction from FBN. Previous GCN studies primarily emphasized the exploration of topological simultaneously connection weights of the estimated FBNs while only focusing on the single connection pattern. However, this approach fails to exploit the potential complementary information offered by different connection patterns of FBNs, thereby inherently limiting the performance. To enhance the diagnostic performance, we propose a multipattern graph convolution network (MPGCN) that integrates multiple connection patterns to improve the accuracy of ASD diagnosis. As an initial endeavor, we endeavored to integrate information from multiple connection patterns by incorporating multiple graph convolution modules. The effectiveness of the MPGCN approach is evaluated by analyzing rs-fMRI scans from a cohort of 92 subjects sourced from the publicly accessible Autism Brain Imaging Data Exchange database. Notably, the experiment demonstrates that our model achieves an accuracy of 91.1% and an area under ROC curve score of 0.9742. The implementation codes are available at https://github.com/immutableJackz/MPGCN.

Which Link Matters? Maintaining Connectivity of Uncertain Networks Under Adversarial Attack

This paper studies connectivity maintenance in uncertain networks under adversarial attack, where a defender conceals crucial links to prevent the largest connected component from being decomposed by an attacker. In contrast with its static counterpart, connectivity maintenance in uncertain networks involves additional probing on links to determine their existence. Therefore, by modeling an uncertain network as a random graph with each link associated with an existence probability and a probing cost, our goal is to design a defensive strategy for link selection that maximizes the expected size of the largest remaining connected component with the minimum expected probing cost, and moreover, the strategy should be independent of the attacking patterns. To this end, we first unravel the computational complexity of the problem by proving its NP-hardness, and then propose optimal defensive strategies based on dynamic programming and multi-objective optimization. Due to the prohibitive computational cost of optimality, two approximate defensive strategies are further designed to pursue decent performance with quasilinear complexity, in which the first one is a heuristic approach that quantifies the link vulnerability through an analogy from the degree centrality of a vertex in static networks to the connectivity weight of a link in uncertain networks, and the second one is an adaptive greedy policy incorporating the minimax rule from game theory, which minimizes the possible loss suffered by the defender in a worst-case scenario and has a constant approximation ratio. Extensive experiments on both synthetic and real-world network datasets under diverse attacking patterns demonstrate the superiority of the proposed strategies over baselines.

An exploratory computational analysis in mice brain networks of widespread epileptic seizure onset locations along with potential strategies for effective intervention and propagation control.

Mean-field models have been developed to replicate key features of epileptic seizure dynamics. However, the precise mechanisms and the role of the brain area responsible for seizure onset and propagation remain incompletely understood. In this study, we employ computational methods within The Virtual Brain framework and the Epileptor model to explore how the location and connectivity of an Epileptogenic Zone (EZ) in a mouse brain are related to focal seizures (seizures that start in one brain area and may or may not remain localized), with a specific focus on the hippocampal region known for its association with epileptic seizures. We then devise computational strategies to confine seizures (prevent widespread propagation), simulating medical-like treatments such as tissue resection and the application of an anti-seizure drugs or neurostimulation to suppress hyperexcitability. Through selectively removing (blocking) specific connections informed by the structural connectome and graph network measurements or by locally reducing outgoing connection weights of EZ areas, we demonstrate that seizures can be kept constrained around the EZ region. We successfully identified the minimal connections necessary to prevent widespread seizures, with a particular focus on minimizing surgical or medical intervention while simultaneously preserving the original structural connectivity and maximizing brain functionality.

Impact of non-diffusive interactions on Turing instability

Observational studies of the chemical reactions have shown that the uniform state, driven by the diffusion differences of activator and inhibitor species, can seed the spontaneous emergence of spatially organized patterns—also known as Turing instability. The presence of diffusive interaction between the species in discrete media can naturally form the structure of the network Laplacian, thereby resulting in that the Turing instability can be better understood via the eigenvectors and eigenvalues of the graph Laplacian. However, it is still unclear whether and under which conditions the Turing instability can be induced in the absence of diffusive transport. In this work, to overcome the limitation of network Laplacian matrix that only has non-positive eigenvalues, we investigate the effects of non-diffusive (or no-local) coupling between the species on the Turing instability for the directed and undirected networks, respectively, where the interaction matrix may produce the positive and negative eigenvalues. In particular, we first derive the critical conditions that the Turing instability occurs by employing the linear stability analysis. Then we apply them to the networked systems governed by the FitzHugh–Nagumo dynamics and obtain the critical range of eigenvalues of the interaction matrix. Finally, we use the Girko’s circular law to predict the eigenvalues’ distribution, and show, via extensive numerical simulations, the transition process of the Turing instability depending on network size, network connectivity and connection weights (intraspecific interaction strength). Our results reveal that in the absence of diffusive transport, the Turing instability still can arise, offering a new perspective on the generation mechanism of the Turing instability.

Quantifying and Maximizing the Information Flux in Recurrent Neural Networks.

Free-running recurrent neural networks (RNNs), especially probabilistic models, generate an ongoing information flux that can be quantified with the mutual information I[x→(t),x→(t+1)] between subsequent system states x→. Although previous studies have shown that I depends on the statistics of the network's connection weights, it is unclear how to maximize I systematically and how to quantify the flux in large systems where computing the mutual information becomes intractable. Here, we address these questions using Boltzmann machines as model systems. We find that in networks with moderately strong connections, the mutual information I is approximately a monotonic transformation of the root-mean-square averaged Pearson correlations between neuron pairs, a quantity that can be efficiently computed even in large systems. Furthermore, evolutionary maximization of I[x→(t),x→(t+1)] reveals a general design principle for the weight matrices enabling the systematic construction of systems with a high spontaneous information flux. Finally, we simultaneously maximize information flux and the mean period length of cyclic attractors in the state-space of these dynamical networks. Our results are potentially useful for the construction of RNNs that serve as short-time memories or pattern generators.

Silver Nanowire Networks with Moisture-Enhanced Learning Ability.

The human brain possesses a remarkable ability to memorize information with the assistance of a specific external environment. Therefore, mimicking the human brain's environment-enhanced learning abilities in artificial electronic devices is essential but remains a considerable challenge. Here, a network of Ag nanowires with a moisture-enhanced learning ability, which can mimic long-term potentiation (LTP) synaptic plasticity at an ultralow operating voltage as low as 0.01 V, is presented. To realize a moisture-enhanced learning ability and to adjust the aggregations of Ag ions, we introduced a thin polyvinylpyrrolidone (PVP) coating layer with moisture-sensitive properties to the surfaces of the Ag nanowires of Ag ions. That Ag nanowire network was shown to exhibit, in response to the humidity of its operating environment, different learning speeds during the LTP process. In high-humidity environments, the synaptic plasticity was significantly strengthened with a higher learning speed compared with that in relatively low-humidity environments. Based on experimental and simulation results, we attribute this enhancement to the higher electric mobility of the Ag ions in the water-absorbed PVP layer. Finally, we demonstrated by simulation that the moisture-enhanced synaptic plasticity enabled the device to adjust connection weights and delivery modes based on various input patterns. The recognition rate of a handwritten data set reached 94.5% with fewer epochs in a high-humidity environment. This work shows the feasibility of building our electronic device to achieve artificial adaptive learning abilities.

Interval Type-2 Mutual Subsethood Cauchy Fuzzy Neural Inference System (IT2MSCFuNIS)

An interval type-2 (IT2) mutual subsethood Cauchy fuzzy neural inference system has been proposed in this paper. The network architecture consists of 3-layers with all connection weights being IT2 Cauchy fuzzy membership functions (CMFs). The crisp inputs to the system are fuzzified into IT2CMFs with fixed centers and uncertain spreads. The hidden layer represents the rule-based knowledge. The firing degree of the antecedent part of each rule at the hidden layer is computed by aggregating the product of the mutual subsethood similarity measures between the inputs and the connection weights. A volume defuzzification is used to compute the numeric output. A gradient descent back-propagation algorithm is used to train the model. The novelty of the proposed model is threefold. First, is enriching the theory of the mutual subsethood fuzzy neural models by adopting the Cauchy membership function (CMF) as another powerful fuzzy basis function (FBF) rather than the classical choice of Gaussian fuzzy membership functions (GMFs). Second, is the success of computing the mutual subsethood similarity measure between the IT2CMFs and all the model parameters’ updating equations in analytic closed-form formulas, not numerically or approximately. Third, is the ability to extract the type-1 (T1) mutual subsethood Cauchy fuzzy neural inference system (T1MSCFuNIS) with all its analytic closed-form formulas directly as a special case from the general formulas of IT2MSCFuNIS model. Such a novelty makes the proposed model a concrete and effective development of the theory of mutual subsethood fuzzy neural models. Both IT2MSCFuNIS and T1MSCFuNIS models have been tested using different examples from the domains of function approximation, classification, and prediction. The results ensure the efficacy of both models compared with other models reported in the literature.

Full-function Pavlov associative learning photonic neural networks based on SOA and DFB-SA

Pavlovian associative learning, a form of classical conditioning, has significantly impacted the development of psychology and neuroscience. However, the realization of a prototypical photonic neural network (PNN) for full-function Pavlov associative learning, encompassing both photonic synapses and photonic neurons, has not been achieved to date. In this study, we propose and experimentally demonstrate the first InP-based full-function Pavlov associative learning PNN. The PNN utilizes semiconductor optical amplifiers (SOAs) as photonic synapses and the distributed feedback laser with a saturable absorber (DFB-SA) as the photonic spiking neuron. The connection weights between neurons in the PNN can be dynamically changed based on the fast, time-varying weighting properties of the SOA. The optical output of the SOA can be directly coupled into the DFB-SA laser for nonlinear computation without additional photoelectric conversion. The results indicate that the PNN can successfully perform brain-like computing functions such as associative learning, forgetting, and pattern recall. Furthermore, we analyze the performance of PNN in terms of speed, energy consumption, bandwidth, and cascadability. A computational model of the PNN is derived based on the distributed time-domain coupled traveling wave equations. The numerical results agree well with the experimental findings. The proposed full-function Pavlovian associative learning PNN is expected to play an important role in the development of the field of photonic brain-like neuromorphic computing.

Evolving Dual-Threshold Bienenstock-Cooper-Munro Learning Rules in Echo State Networks.

The strengthening and the weakening of synaptic strength in existing Bienenstock-Cooper-Munro (BCM) learning rule are determined by a long-term potentiation (LTP) sliding modification threshold and the afferent synaptic activities. However, synaptic long-term depression (LTD) even affects low-active synapses during the induction of synaptic plasticity, which may lead to information loss. Biological experiments have found another LTD threshold that can induce either potentiation or depression or no change, even at the activated synapses. In addition, existing BCM learning rules can only select a set of fixed rule parameters, which is biologically implausible and practically inflexible to learn the structural information of input signals. In this article, an evolved dual-threshold BCM learning rule is proposed to regulate the reservoir internal connection weights of the echo-state-network (ESN), which can contribute to alleviating information loss and enhancing learning performance by introducing different optimal LTD thresholds for different postsynaptic neurons. Our experimental results show that the evolved dual-threshold BCM learning rule can result in the synergistic learning of different plasticity rules, effectively improving the learning performance of an ESN in comparison with existing neural plasticity learning rules and some state-of-the-art ESN variants on three widely used benchmark tasks and the prediction of an esterification process.

Forecasting the Nexus and Impact of News Sentiment on NYSE, Gold Prices, and WTI Oil Using The Neural Network Approach

The study aims to determine the impact of news sentiment during COVID-19 and the nexus between the NYSE, Gold prices, and WTI oil prices. For this purpose, we use hourly data of news sentiment and commodity prices hourly data from February 10, 2020, hours 1000 to March 06, 2020 hours 1600 hours, with the help of a multilayer artificial neural network. Further, We use the connection weight approach to observe the relative importance of input variables that help predict output variables. The results suggest that all the indexes are interconnected, where the increase/decrease of any index impacts the other commodity index. Further, the news index remains the second highest predictor for gold price and WTI oil price indexes. In addition, the news index also seems to be an essential predictor of these commodities.
 Keywords: NYSE, WTI Oil, Gold, Covid-19, Neural Network, Connection Weight