Connection Weights Research Articles

Investigation on an inversion method of ocean salinity by lidar based on a neural network

To achieve high-precision and high-speed inversion of salinity using lidar data, this paper proposes a method for inversion of salinity based on back propagation neural network. Both the Raman spectra and the Brillouin frequency shift are related to temperature and salinity of ocean. However, pressure, refractive index, sound velocity, suspended particles, colored dissolved organic matter and ocean currents will affect the detection results when using lidar detection. Therefore, we establish a neural network model with 8 input parameters (NN-8), 5 input parameters (NN-5), and 2 input parameters (NN-2) and compares the inversion results of three models, NN-8 and NN-5 with higher accuracy, analyzes the correlation between the measurement results and the inversion results. The correlation coefficients of both NN-8 and NN-5 are greater than 0.999. Their RMSEs are 0.203‰ and 0.205‰, and REs are 0.09% and 0.105%, respectively. NN-5 can achieve higher inversion precision with fewer parameters. The influence of detection parameters on salinity inversion is evaluated using connection weights. Brillouin frequency shift and Raman spectra play a leading role in the salinity inversion, and other influencing factors can help to improve the inversion accuracy. Raman spectra at different salinities were detected experimentally, and the results are verified using the established model. The model results are consistent with the experimental results, and the error is less than 0.4‰. Compared with other classical regression methods, the results showed that the inversion results have high accuracy, and their errors are all less than 0.4‰. The research can offer data support for the global climates and ecosystems.

Toward Deep Adaptive Hinging Hyperplanes.

The adaptive hinging hyperplane (AHH) model is a popular piecewise linear representation with a generalized tree structure and has been successfully applied in dynamic system identification. In this article, we aim to construct the deep AHH (DAHH) model to extend and generalize the networking of AHH model for high-dimensional problems. The network structure of DAHH is determined through a forward growth, in which the activity ratio is introduced to select effective neurons and no connecting weights are involved between the layers. Then, all neurons in the DAHH network can be flexibly connected to the output in a skip-layer format, and only the corresponding weights are the parameters to optimize. With such a network framework, the backpropagation algorithm can be implemented in DAHH to efficiently tackle large-scale problems and the gradient vanishing problem is not encountered in the training of DAHH. In fact, the optimization problem of DAHH can maintain convexity with convex loss in the output layer, which brings natural advantages in optimization. Different from the existing neural networks, DAHH is easier to interpret, where neurons are connected sparsely and analysis of variance (ANOVA) decomposition can be applied, facilitating to revealing the interactions between variables. A theoretical analysis toward universal approximation ability and explicit domain partitions are also derived. Numerical experiments verify the effectiveness of the proposed DAHH.

Graphop mean-field limits and synchronization for the stochastic Kuramoto model

Models of coupled oscillator networks play an important role in describing collective synchronization dynamics in biological and technological systems. The Kuramoto model describes oscillator's phase evolution and explains the transition from incoherent to coherent oscillations under simplifying assumptions, including all-to-all coupling with uniform strength. Real world networks, however, often display heterogeneous connectivity and coupling weights that influence the critical threshold for this transition. We formulate a general mean-field theory (Vlasov-Focker Planck equation) for stochastic Kuramoto-type phase oscillator models, valid for coupling graphs/networks with heterogeneous connectivity and coupling strengths, using graphop theory in the mean-field limit. Considering symmetric odd-valued coupling functions, we mathematically prove an exact formula for the critical threshold for the incoherence-coherence transition. We numerically test the predicted threshold using large finite-size representations of the network model. For a large class of graph models, we find that the numerical tests agree very well with the predicted threshold obtained from mean-field theory. However, the prediction is more difficult in practice for graph structures that are sufficiently sparse. Our findings open future research avenues toward a deeper understanding of mean-field theories for heterogeneous systems.

Scalable Gamma-Driven Multilayer Network for Brain Workload Detection Through Functional Near-Infrared Spectroscopy.

This work proposes a scalable gamma non-negative matrix network (SGNMN), which uses a Poisson randomized Gamma factor analysis to obtain the neurons of the first layer of a network. These neurons obey Gamma distribution whose shape parameter infers the neurons of the next layer of the network and their related weights. Upsampling the connection weights follows a Dirichlet distribution. Downsampling hidden units obey Gamma distribution. This work performs up-down sampling on each layer to learn the parameters of SGNMN. Experimental results indicate that the width and depth of SGNMN are closely related, and a reasonable network structure for accurately detecting brain fatigue through functional near-infrared spectroscopy can be obtained by considering network width, depth, and parameters.

Reaction-diffusion models in weighted and directed connectomes.

Connectomes represent comprehensive descriptions of neural connections in a nervous system to better understand and model central brain function and peripheral processing of afferent and efferent neural signals. Connectomes can be considered as a distinctive and necessary structural component alongside glial, vascular, neurochemical, and metabolic networks of the nervous systems of higher organisms that are required for the control of body functions and interaction with the environment. They are carriers of functional phenomena such as planning behavior and cognition, which are based on the processing of highly dynamic neural signaling patterns. In this study, we examine more detailed connectomes with edge weighting and orientation properties, in which reciprocal neuronal connections are also considered. Diffusion processes are a further necessary condition for generating dynamic bioelectric patterns in connectomes. Based on our precise connectome data, we investigate different diffusion-reaction models to study the propagation of dynamic concentration patterns in control and lesioned connectomes. Therefore, differential equations for modeling diffusion were combined with well-known reaction terms to allow the use of connection weights, connectivity orientation and spatial distances. Three reaction-diffusion systems Gray-Scott, Gierer-Meinhardt and Mimura-Murray were investigated. For this purpose, implicit solvers were implemented in a numerically stable reaction-diffusion system within the framework of neuroVIISAS. The implemented reaction-diffusion systems were applied to a subconnectome which shapes the mechanosensitive pathway that is strongly affected in the multiple sclerosis demyelination disease. It was found that demyelination modeling by connectivity weight modulation changes the oscillations of the target region, i.e. the primary somatosensory cortex, of the mechanosensitive pathway. In conclusion, a new application of reaction-diffusion systems to weighted and directed connectomes has been realized. Because the implementation was realized in the neuroVIISAS framework many possibilities for the study of dynamic reaction-diffusion processes in empirical connectomes as well as specific randomized network models are available now.

Error driven synapse augmented neurogenesis.

Capturing the learning capabilities of the brain has the potential to revolutionize artificial intelligence. Humans display an impressive ability to acquire knowledge on the fly and immediately store it in a usable format. Parametric models of learning, such as gradient descent, focus on capturing the statistical properties of a data set. Information is precipitated into a network through repeated updates of connection weights in the direction gradients dictate will lead to less error. This work presents the EDN (Error Driven Neurogenesis) algorithm which explores how neurogenesis coupled with non-linear synaptic activations enables a biologically plausible mechanism to immediately store data in a one-shot, online fashion and readily apply it to a task without the need for parameter updates. Regression (auto-mpg) test error was reduced more than 135 times faster and converged to an error around three times smaller compared to gradient descent using ADAM optimization. EDN also reached the same level of performance in wine cultivar classification 25 times faster than gradient descent and twice as fast when applied to MNIST and the inverted pendulum (reinforcement learning).

Simple and Effective Fault Diagnosis Method of Power Lithium-Ion Battery Based on GWA-DBN

Abstract In order to improve the accuracy of battery pack inconsistency fault detection, an optimal deep belief network (DBN) single battery inconsistency fault detection model based on the gray wolf algorithm (GWA) was proposed. The performance of the DBN model is affected by the weights and bias parameters, and the gray wolf algorithm has a good ability to seek optimization, so the gray wolf algorithm is used to optimize the connection weights of the DBN model. Therefore, the accuracy rate of battery inconsistency diagnosis is improved. The battery voltage characteristic data is used as the input signal of the DBN model. The health and faults of the single cells are used as the output signals of the DBN model. The battery inconsistency fault detection model of GWA-DBN is established. Through the comparison and simulation with other algorithms, it is proved that the designed model has higher diagnostic accuracy, better fitting effect, and good application prospect.

The relative importance of textual indexes in predicting the future performance of banks: A connection weight approach

We employ a neural network approach to predict banks' future performance with the help of textual information and financial data. Neural network models are considered black boxes because they do not help explain the role of each input variable in predicting the output variable. Therefore, the study uses the connection weight approach to identify the relative importance of textual indexes to represent the sentiments of management in the annual reports of 62 banks from 2007 to 2018. The results of the connection weight approach suggest that several textual indexes are very important for predicting the future financial performance of banks. This study is the first of its kind by suggesting that a better predictive model (neural network) needs to be built not only on bank-level financial variables but also include textual information, which is very informative for performance prediction.

Parallel and deep reservoir computing using semiconductor lasers with optical feedback

Abstract Photonic reservoir computing has been intensively investigated to solve machine learning tasks effectively. A simple learning procedure of output weights is used for reservoir computing. However, the lack of training of input-node and inter-node connection weights limits the performance of reservoir computing. The use of multiple reservoirs can be a solution to overcome this limitation of reservoir computing. In this study, we investigate parallel and deep configurations of delay-based all-optical reservoir computing using semiconductor lasers with optical feedback by combining multiple reservoirs to improve the performance of reservoir computing. Furthermore, we propose a hybrid configuration to maximize the benefits of parallel and deep reservoirs. We perform the chaotic time-series prediction task, nonlinear channel equalization task, and memory capacity measurement. Then, we compare the performance of single, parallel, deep, and hybrid reservoir configurations. We find that deep reservoirs are suitable for a chaotic time-series prediction task, whereas parallel reservoirs are suitable for a nonlinear channel equalization task. Hybrid reservoirs outperform other configurations for all three tasks. We further optimize the number of reservoirs for each reservoir configuration. Multiple reservoirs show great potential for the improvement of reservoir computing, which in turn can be applied for high-performance edge computing.

The Association of Social Connectivity and Body Weight With the Onset of Type 2 Diabetes: Findings From the Population-Based Prospective MONICA/KORA Cohort.

Low levels of social connectivity are related to the onset of type 2 diabetes mellitus (T2D), and this study investigates the role of body weight in this association. In a sample of 9448 participants followed for a mean of 15.3 years (186,158.5 person-years) from the Monitoring of Trends and Determinants in Cardiovascular Disease Augsburg/Cooperative Health Research in the Region of Augsburg population-based cohort conducted in Germany, we investigated the association of social connectivity, measured by the Social Network Index, and body mass index (BMI) with the risk of clinically validated T2D incidence using stratified Cox proportional hazards regression models adjusted for sociodemographic, life-style, cardiometabolic, and psychosocial risk factors. During a mean follow-up of 14.1 years (186,158.5 person-years), 975 (10.3%) participants developed T2D. Participants with low social connectivity developed T2D at a higher rate than socially connected participants (10.0 versus 8.0 cases/10,000 person-years); however, BMI played a significant role in the association of social connectivity with T2D ( p < .001). In comparison to their socially connected counterparts, low social connectivity was associated with a higher rate of T2D incidence in normal-weight (6.0 versus 2.0 cases/10,000 person-years), but not overweight (13.0 versus 13.0 cases/10,000 person-years) or obese participants (32.0 versus 30.0 cases/10,000 person-years). Correspondingly, Cox regression analysis showed that 5-unit increments in BMI increased the risk of T2D in socially connected participants (hazard ratio = 3.03, 95% confidence interval = 2.48-3.79, p < .001) at a substantially higher rate than in low socially connected participants (hazard ratio = 1.77, 95% confidence interval = 1.45-2.16, p < .001). The detrimental link between low social connectivity and increased risk of T2D is substantially stronger in participants with a lower BMI.

Hebbian, correlational learning provides a memory-less mechanism for Statistical Learning irrespective of implementational choices: Reply to Tovar and Westermann (2022)

Statistical learning relies on detecting the frequency of co-occurrences of items and has been proposed to be crucial for a variety of learning problems, notably to learn and memorize words from fluent speech. Endress and Johnson (2021) (hereafter EJ) recently showed that such results can be explained based on simple memory-less correlational learning mechanisms such as Hebbian Learning. Tovar and Westermann (2022) (hereafter TW) reproduced these results with a different Hebbian model. We show that the main differences between the models are whether temporal decay acts on both the connection weights and the activations (in TW) or only on the activations (in EJ), and whether interference affects weights (in TW) or activations (in EJ). Given that weights and activations are linked through the Hebbian learning rule, the networks behave similarly. However, in contrast to TW, we do not believe that neurophysiological data are relevant to adjudicate between abstract psychological models with little biological detail. Taken together, both models show that different memory-less correlational learning mechanisms provide a parsimonious account of Statistical Learning results. They are consistent with evidence that Statistical Learning might not allow learners to learn and retain words, and Statistical Learning might support predictive processing instead.

In vivo extracellular recordings of thalamic and cortical visual responses reveal V1 connectivity rules

The brain's connectome provides the scaffold for canonical neural computations. However, a comparison of connectivity studies in the mouse primary visual cortex (V1) reveals that the average number and strength of connections between specific neuron types can vary. Can variability in V1 connectivity measurements coexist with canonical neural computations? We developed a theory-driven approach to deduce V1 network connectivity from visual responses in mouse V1 and visual thalamus (dLGN). Our method revealed that the same recorded visual responses were captured by multiple connectivity configurations. Remarkably, the magnitude and selectivity of connectivity weights followed a specific order across most of the inferred connectivity configurations. We argue that this order stems from the specific shapes of the recorded contrast response functions and contrast invariance of orientation tuning. Remarkably, despite variability across connectivity studies, connectivity weights computed from individual published connectivity reports followed the order we identified with our method, suggesting that the relations between the weights, rather than their magnitudes, represent a connectivity motif supporting canonical V1 computations.

Lattice connectivity and entanglement in quantum spin-glasses

I have studied the role of lattice connectivity and coupling weights distribution on the entanglement of quantum spin-glasses. It's found in this work that the connectivity of the lattice weakly influence the degree of entanglement in the spin-glass compared to the distribution of the coupling constants between the spins. This suggest important implications for machine learning models such as Boltzmann machines and the study of complex quantum systems.

Self-organizing map with granular competitive learning: Application to microarray clustering

Self-organizing map (SOM) models perform clustering process based on a competitive learning. The learning methods of these models involve neighborhood function such as Gaussian in the output layer, where the Euclidean distance from winning node to an output node is used. In this study, a granular competitive learning of SOM (SOMGCL) involving a fuzzy distance, the distance based granular neighborhood function and fuzzy initial connection weights is developed using the concepts of fuzzy rough set. The fuzzy distance between a winning node and an output node of SOM is computed where the average of memberships belonging to the lower approximations and boundary regions of a cluster obtained at the node is used. The fuzzy distance is incorporated into a Gaussian function to define the proposed neighborhood function. Dependency values of features using fuzzy rough sets are encoded into SOM as its fuzzy initial connection weights. Here, the concepts of fuzzy rough set are based on a new fuzzy strict order relation. While the fuzzy distance defines similarity measure in clustering process, the distance based granular neighborhood function handles uncertainty in cluster boundary regions. The effectiveness of SOMGCL is demonstrated in clustering of both the samples and genes in microarrays having the large number of genes and classes in terms of cluster evaluation metrics and quantization error. Further, biological meaning of gene clusters obtained using SOMGCL is elucidated using gene-ontology.

Multistrategy Improved Sparrow Search Algorithm Optimized Deep Neural Network for Esophageal Cancer.

Deep neural network is a complex pattern recognition network system. It is widely favored by scholars for its strong nonlinear fitting ability. However, training deep neural network models on small datasets typically realizes worse performance than shallow neural network. In this study, a strategy to improve the sparrow search algorithm based on the iterative map, iterative perturbation, and Gaussian mutation is developed. This optimized strategy improved the sparrow search algorithm validated by fourteen benchmark functions, and the algorithm has the best search accuracy and the fastest convergence speed. An algorithm based on the iterative map, iterative perturbation, and Gaussian mutation improved sparrow search algorithm is designed to optimize deep neural networks. The modified sparrow algorithm is exploited to search for the optimal connection weights of deep neural network. This algorithm is implemented for the esophageal cancer dataset along with the other six algorithms. The proposed model is able to achieve 0.92 under all the eight scoring criteria, which is better than the performance of the other six algorithms. Therefore, an optimized deep neural network based on an improved sparrow search algorithm with iterative map, iterative perturbation, and Gaussian mutation is an effective approach to predict the survival rate of esophageal cancer.

Estimation of the compaction parameters of aggregate base course using artificial neural networks

The process of estimating the compaction parameters namely the maximum dry density (MDD) and optimum moisture content (OMC) through laboratory tests is time-consuming, labor-intensive, and costly. These issues can be avoided by developing prediction models that are able to accurately predict the compaction parameters from index properties that are easier to estimate in the laboratory. As a result, this study focuses on employing artificial neural networks (ANNs) for the prediction of the compaction parameters of aggregate base course samples from the grain size distribution and Atterberg limits. Additionally, different ANNs with different structures were tested in order to set the optimum hyperparameters that minimize the errors in the predictions. Specifically, this study investigates the impact of the number of hidden layers, number of neurons per hidden layer, and activation functions on the performance of the ANNs. Furthermore, the weight decay method, which is the most common regularization technique, was used during the training of the ANNs in order to avoid overfitting and control the changes in the connection weights. The results indicate that the optimum hyperparameter settings changes depending on the optimized output. Additionally, the ReLU activation function is the most stable function that produces the best predictions. Moreover, the results show that ANN approach represents a major innovative tool for accurately predicting the compaction parameters with R2 values of 0.826 and 0.911 for predicting the MDD and OMC.

Recognition ability of untrained neural networks to symbolic numbers.

Although animals can learn to use abstract numbers to represent the number of items, whether untrained animals could distinguish between different abstract numbers is not clear. A two-layer spiking neural network with lateral inhibition was built from the perspective of biological interpretability. The network connection weight was set randomly without adjustment. On the basis of this model, experiments were carried out on the symbolic number dataset MNIST and non-symbolic numerosity dataset. Results showed that the model has abilities to distinguish symbolic numbers. However, compared with number sense, tuning curves of symbolic numbers could not reproduce size and distance effects. The preference distribution also could not show high distribution characteristics at both ends and low distribution characteristics in the middle. More than half of the network units prefer the symbolic numbers 0 and 5. The average goodness-of-fit of the Gaussian fitting of tuning curves increases with the increase in abscissa non-linearity. These results revealed that the concept of human symbolic number is trained on the basis of number sense.

Classification of Program Texts Represented as Markov Chains with Biology-Inspired Algorithms-Enhanced Extreme Learning Machines

The massive nature of modern university programming courses increases the burden on academic workers. The Digital Teaching Assistant (DTA) system addresses this issue by automating unique programming exercise generation and checking, and provides means for analyzing programs received from students by the end of semester. In this paper, we propose a machine learning-based approach to the classification of student programs represented as Markov chains. The proposed approach enables real-time student submissions analysis in the DTA system. We compare the performance of different multi-class classification algorithms, such as support vector machine (SVM), the k nearest neighbors (KNN) algorithm, random forest (RF), and extreme learning machine (ELM). ELM is a single-hidden layer feedforward network (SLFN) learning scheme that drastically speeds up the SLFN training process. This is achieved by randomly initializing weights of connections among input and hidden neurons, and explicitly computing weights of connections among hidden and output neurons. The experimental results show that ELM is the most computationally efficient algorithm among the considered ones. In addition, we apply biology-inspired algorithms to ELM input weights fine-tuning in order to further improve the generalization capabilities of this algorithm. The obtained results show that ELMs fine-tuned with biology-inspired algorithms achieve the best accuracy on test data in most of the considered problems.

Optimization of Residential Landscape Design and Supply Chain System Using Intelligent Fuzzy Cognitive Map and Genetic Algorithm.

This work intends to optimize residential landscape design and Supply Chain (SC) network systems. First, Fuzzy Cognitive Map (FCM) intelligent assistance and genetic algorithm (GA) are used to study residential landscape design and its integration with SC deeply. Weight matrix interactions are employed to implement iterative inference for FCM. The functions are transformed to unify variables of different scopes. Subsequently, a weighting method is proposed to deal with the disadvantage of the simple average method being too general. In addition, the Hebbian learning algorithm is used to adjust the state nodes and the connection weights. Finally, according to the fitness function of the GA and logistic regression (LR) model, residential landscape design and SC are combined. The simulation experiment results show that the causal relationship analysis between SC networks under fuzzy cognition shows that the state errors of each specific situation are 0.21, 0.16, and 0.24, respectively. The total average error is 0.21 in the case of multiple iterations. The average error of the result vector under fuzzy cognition and the operation of the actual result is 0.20, 0.15, and 0.24, respectively, and the error value is much reduced. The simulation accuracy of the GA-LR method for residential landscape design is improved from 77% to 84.7%. The “kappa coefficient” is also improved to 82.3%. The conclusion shows that the weight matrix is used to analyze the high-quality performance of landscape design according to the specific situation of SC. For each specific case, FCM is effective in reducing errors over multiple iterations. Under the GA-LR method, fewer geographic location types and larger accuracy deviations can improve the simulation accuracy.

Global Network Analysis of Alzheimer's Disease with Minimum Spanning Trees.

A minimum spanning tree (MST) is a unique efficient network comprising the necessary connections needed to connect all regions in a network while retaining the lowest possible cost of connection weight. This study aimed to utilize functional near-infrared spectroscopy (fNIRS) to analyze brain activity in different regions and then construct MST-based regions to characterize the brain topologies of participants with Alzheimer's disease (AD), mild cognitive impairment (MCI), and normal controls (NC). A 46 channel fNIRS setup was used on all participants, with correlation being calculated for each channel pair. An MST was constructed from the resulting correlation matrix, from which graph theory measures were calculated. The average number of connections within a lobe in the left versus right hemisphere was calculated to identify which lobes displayed and abnormal amount of connectivity. Compared to those in the MCI group, the AD group showed a less integrated network structure, with a higher characteristic path length, but lower leaf fraction, maximum degree, and degree divergence. The AD group also showed a higher number of connections in the frontal lobe within the left hemisphere and a lower number between hemispheric frontal lobes as compared to MCI. These results indicate a deviation in network structure and connectivity within patient groups that is consistent with the theory of dysconnectivity for AD. Additionally, the AD group showed strong correlations between the Hamilton depression rating scale and different graph metrics, suggesting a link between network organization and the recurrence of depression in AD.