Connection Weights Of Network Research Articles

Deep multi-sphere support vector data description based on disentangled representation learning

Deep support vector data description (Deep SVDD) combines deep mapping network and support vector data description (SVDD) to jointly optimize network connection weights and hypersphere volume. However, when the parameters of deep mapping network are set improperly, Deep SVDD may face the problem of hypersphere collapse, where all input data are mapped as the hypersphere center. To overcome the hypersphere collapse problem of Deep SVDD and improve the feature learning ability of deep mapping network, deep multi-sphere SVDD based on disentangled representation learning (DMSVDD-DRL) is proposed. DMSVDD-DRL consists of a variational autoencoder (VAE) and multiple hyperspheres. The feature representations obtained by VAE are disentangled into discriminative representations and generative representations that obey mixture t-distribution and Gaussian distribution, respectively. In the pre-training phase of DMSVDD-DRL, the network parameters and the hypersphere centers are initialized. In the training phase, the augmented data are added into the training set. The discriminative representations of both the input and augmented data are generated through the mapping network. Furthermore, multiple hyperspheres are constructed by the obtained discriminative representations in the feature space. Finally, the VAE loss of the input data, the reconstruction error of the augmented data, the augmentation loss between the input and augmented data, the average radius of the multiple hyperspheres, and the average distance from discriminative representations to their corresponding hypersphere centers are jointly minimized to obtain the optimal network connection weights and the multiple minimum volume hyperspheres. The effectiveness of the proposed DMSVDD-DRL is validated through the comparative and ablation experiments on the benchmark data sets. In addition, it is verified that DMSVDD-DRL is more robust against outliers in comparison with its related methods.

Quantifying and Maximizing the Information Flux in Recurrent Neural Networks.

Free-running recurrent neural networks (RNNs), especially probabilistic models, generate an ongoing information flux that can be quantified with the mutual information I[x→(t),x→(t+1)] between subsequent system states x→. Although previous studies have shown that I depends on the statistics of the network's connection weights, it is unclear how to maximize I systematically and how to quantify the flux in large systems where computing the mutual information becomes intractable. Here, we address these questions using Boltzmann machines as model systems. We find that in networks with moderately strong connections, the mutual information I is approximately a monotonic transformation of the root-mean-square averaged Pearson correlations between neuron pairs, a quantity that can be efficiently computed even in large systems. Furthermore, evolutionary maximization of I[x→(t),x→(t+1)] reveals a general design principle for the weight matrices enabling the systematic construction of systems with a high spontaneous information flux. Finally, we simultaneously maximize information flux and the mean period length of cyclic attractors in the state-space of these dynamical networks. Our results are potentially useful for the construction of RNNs that serve as short-time memories or pattern generators.

Prediction model of spontaneous combustion risk of extraction borehole based on PSO-BPNN and its application

The feasibility and accuracy of the risk prediction of gas extraction borehole spontaneous combustion is improved to avoid the occurrence of spontaneous combustion in the gas extraction borehole. A gas extraction borehole spontaneous combustion risk prediction model (PSO-BPNN model) coupling the PSO algorithm with BP neural network is established through improving the connection weight and threshold values of BP neural network by the particle swarm optimization (PSO) algorithm. The prediction results of the PSO-BPNN model are compared and analyzed with that of the BP neural network model (BPNN model), GA-BPNN model, SSA-BPNN model and MPA-BPNN model. The results showed as follows: the average relative error of the PSO-BPNN model was 4.38%; the average absolute error was 0.0678; the root mean square error was 0.0934; and the determination coefficient was 0.9874. Compared with the BPNN model, the average relative error, average absolute error and root mean square error decreased by 9.35%, 0.1707 and 0.2056 respectively; and the determination coefficient increased by 0.1169. Compared with the GA-BPNN model, the average relative error, average absolute error and root mean square error decreased by 3.19%, 0.0602 and 0.0821 respectively; and the determination coefficient increased by 0.0320. Compared with the SSA-BPNN model, the average relative error, average absolute error and root mean square error decreased by 5.70%, 0.0820 and 0.1100 respectively; and the determination coefficient increased by 0.0474. Compared with the MPA-BPNN model, the average relative error, average absolute error and root mean square error decreased by 3.50%, 0.0861 and 0.1125 respectively; and the determination coefficient increased by 0.0488, proving that the PSO-BPNN model is more accurate than the BPNN model, GA-BPNN model, SSA-BPNN model and MPA-BPNN model as for prediction. When the PSO-BPNN model was applied to three extraction boreholes A, B, and C in a coal mine of Shanxi, the prediction results were better than the BPNN model, GA-BPNN model, SSA-BPNN model and MPA-BPNN model, proving the accuracy and stability of the PSO-BPNN model in predicting risk of borehole spontaneous combustion in other mine.

Empirical Model for the Retained Stability Index of Asphalt Mixtures Using Hybrid Machine Learning Approach

Moisture susceptibility is a complex phenomenon that induces various distresses in asphalt pavements and can be assessed by the Retained Stability Index (RSI). This study proposes a robust model to predict the RSI using a hybrid machine learning technique, including Artificial Neural Network (ANN) and Gene Expression Programming. The model is expressed as a simple and direct mathematical function with input variables of mineral filler proportion (F%), water absorption rate of combined aggregate (Ab%), asphalt content (AC%), and air void content (Va%). A relative importance analysis ranked AC% as the most influential variable on RSI, followed by Va%, F%, and Ab%. The experimental RSI results of 150 testing samples of various mixes were utilized along with other data points generated by the ANN to train and validate the proposed model. The model promotes a high level of accuracy for predicting the RSI with a 96.6% coefficient of determination (R2) and very low errors. In addition, the sensitivity of the model has been verified by considering the effect of the variables, which is in line with the results of network connection weight and previous studies in the literature. F%, Ab%, and Va% have an inverse relationship with the RSI values, whereas AC% has the opposite. The model helps forecast the water susceptibility of asphalt mixes by which the experimental effort is minimized and the mixes’ performance can be improved.

Generative complex networks within a dynamic memristor with intrinsic variability

Artificial neural networks (ANNs) have gained considerable momentum in the past decade. Although at first the main task of the ANN paradigm was to tune the connection weights in fixed-architecture networks, there has recently been growing interest in evolving network architectures toward the goal of creating artificial general intelligence. Lagging behind this trend, current ANN hardware struggles for a balance between flexibility and efficiency but cannot achieve both. Here, we report on a novel approach for the on-demand generation of complex networks within a single memristor where multiple virtual nodes are created by time multiplexing and the non-trivial topological features, such as small-worldness, are generated by exploiting device dynamics with intrinsic cycle-to-cycle variability. When used for reservoir computing, memristive complex networks can achieve a noticeable increase in memory capacity a and respectable performance boost compared to conventional reservoirs trivially implemented as fully connected networks. This work expands the functionality of memristors for ANN computing.

Incorporating structural plasticity into self-organization recurrent networks for sequence learning.

Spiking neural networks (SNNs), inspired by biological neural networks, have received a surge of interest due to its temporal encoding. Biological neural networks are driven by multiple plasticities, including spike timing-dependent plasticity (STDP), structural plasticity, and homeostatic plasticity, making network connection patterns and weights to change continuously during the lifecycle. However, it is unclear how these plasticities interact to shape neural networks and affect neural signal processing. Here, we propose a reward-modulated self-organization recurrent network with structural plasticity (RSRN-SP) to investigate this issue. Specifically, RSRN-SP uses spikes to encode information, and incorporate multiple plasticities including reward-modulated spike timing-dependent plasticity (R-STDP), homeostatic plasticity, and structural plasticity. On the one hand, combined with homeostatic plasticity, R-STDP is presented to guide the updating of synaptic weights. On the other hand, structural plasticity is utilized to simulate the growth and pruning of synaptic connections. Extensive experiments for sequential learning tasks are conducted to demonstrate the representational ability of the RSRN-SP, including counting task, motion prediction, and motion generation. Furthermore, the simulations also indicate that the characteristics arose from the RSRN-SP are consistent with biological observations.

Contrastive deep support vector data description

In comparison with support vector data description (SVDD), deep SVDD (DSVDD) is more suitable for dealing with large-scale data sets. DSVDD uses mapping network to replace the role of kernel mapping in SVDD. Moreover, the objective of DSVDD is to simultaneously learn the optimal connection weights of mapping network and the minimum volume of hypersphere. To further improve the performance of DSVDD for tackling large-scale data sets and obtain the discriminative features of the given samples in a self-supervised learning manner, contrastive DSVDD (CDSVDD) is proposed in this study. In the pre-training phase of CDSVDD, the contrastive loss and the rotation prediction loss are jointly minimized to achieve the optimal feature representations. Furthermore, the learned feature representations are utilized to determine the hypersphere center. In the training phase of CDSVDD, the distances between the obtained feature representations and the hypersphere center together with the contrastive loss are simultaneously minimized to derive the optimal network connection weights, the minimum volume of hypersphere and the optimal feature representations. In addition, CDSVDD can efficiently solve the hypersphere collapse problem of DSVDD. The ablation study on CDSVDD verifies that compared with the case of determining the hypersphere center by the feature representations of the original samples, the hypersphere center determined by the feature representations of the augmented samples makes CDSVDD achieve better hypersphere boundary and more compact feature representations. Experimental results on the four benchmark data sets demonstrate that the proposed CDSVDD acquires better detection performance in comparison with its six pertinent methods.

A Novel Method for Estimating Accuracy of Network Structure Recovery Model

Many practical systems can be considered as networks of nodes interacting. Explicit network topology is a straightforward method to understand the actual system, so it is of practical significance to obtain the complete network topology from the empirically measured time series. With the premise that the dynamics equations and coupling matrix are known, a method to reconstruct the network topology from the measured time series is proposed, and based on regression theory the estimated matrix form of the adjacency matrix is given. Also, the method is suitable for predicting arbitrary weights of network connections, and its practicality is verified by numerical simulations. Importantly for the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0-1$ </tex-math></inline-formula> matrix, a new method for judging the prediction performance of the model using the false negative rate is proposed. It can estimate the accuracy of model prediction with only partial sampling data when the information of network topology is unknown. In addition, a method that can control false positives is proposed, and the feasibility of the method is verified by numerical simulation. Finally, two factors that affect model performance, the amount of sample data and the intensity of noise, are discussed.

Intelligent Diagnosis Model of Alzheimer's Disease Based on PSO Algorithm Optimized BP Neural Network

In this paper, an intelligent diagnosis model of Alzheimer's disease is established based on the PSO algorithm optimized BP neural network. After descriptive analysis and normalization of patient-specific information characteristic data, the model was established by parameter selection of BP neural network, optimization of BP neural network by PSO algorithm and sensitivity analysis of neural network connection weights, and the intelligent diagnostic model of Alzheimer's disease was tested. The results showed that the model has strong applicability in terms of the degree of Alzheimer's disease in patients, and provides a certain reference basis for accurate diagnosis of the developmental stage of Alzheimer's disease.

Аналіз моделей процесів позаагрегатного оброблення чавуну

The aim of this paper is to perform a generalized analysis of studies on modeling the processes of out-of-furnace treatment of cast iron. Mathematical models are classified according to the basic principles of modeling. A description of different models based on different principles is given, depending on their type and the differences between them. A more detailed analysis of some of the fundamental models and expressions obtained in their construction is carried out. An example of models built on experimental data is given. Neural network models consist of artificial neurons that are connected to each other by means of connecting weights, i.e., model parameters, in the form of layers. Neurons are a set of mathematical functions that modify the input data to obtain an estimate of the desired result. A large number of network parameters makes training a neural network a cumbersome computational process. The large number of network connection weights that need to be optimized when training such models usually requires a large amount of input data. The paper presents domestic achievements in the construction of mathematical models of the out-of-furnace iron treatment process. The principles of creating an integrated database that summarizes information on the parameters of various technologies for desulphurization of cast iron, including the developed system unit of the information retrieval system, are described; the concept of an expert system for making decisions on process control and selection of a rational technology for out-of-furnace desulphurization of cast iron is developed; the variant of the developed information and mathematical support of the expert system with the module for out-of-furnace treatment of cast iron with granular magnesium and coinjection of magnesium and lime is described; the results of the study are presented. The paper describes the models devoted to numerical and physical modeling of the phenomena that occur in the ladle during injection, as well as to the study of the regularities of the gas-powder flow.

Multistrategy Improved Sparrow Search Algorithm Optimized Deep Neural Network for Esophageal Cancer.

Deep neural network is a complex pattern recognition network system. It is widely favored by scholars for its strong nonlinear fitting ability. However, training deep neural network models on small datasets typically realizes worse performance than shallow neural network. In this study, a strategy to improve the sparrow search algorithm based on the iterative map, iterative perturbation, and Gaussian mutation is developed. This optimized strategy improved the sparrow search algorithm validated by fourteen benchmark functions, and the algorithm has the best search accuracy and the fastest convergence speed. An algorithm based on the iterative map, iterative perturbation, and Gaussian mutation improved sparrow search algorithm is designed to optimize deep neural networks. The modified sparrow algorithm is exploited to search for the optimal connection weights of deep neural network. This algorithm is implemented for the esophageal cancer dataset along with the other six algorithms. The proposed model is able to achieve 0.92 under all the eight scoring criteria, which is better than the performance of the other six algorithms. Therefore, an optimized deep neural network based on an improved sparrow search algorithm with iterative map, iterative perturbation, and Gaussian mutation is an effective approach to predict the survival rate of esophageal cancer.

Recognition ability of untrained neural networks to symbolic numbers.

Although animals can learn to use abstract numbers to represent the number of items, whether untrained animals could distinguish between different abstract numbers is not clear. A two-layer spiking neural network with lateral inhibition was built from the perspective of biological interpretability. The network connection weight was set randomly without adjustment. On the basis of this model, experiments were carried out on the symbolic number dataset MNIST and non-symbolic numerosity dataset. Results showed that the model has abilities to distinguish symbolic numbers. However, compared with number sense, tuning curves of symbolic numbers could not reproduce size and distance effects. The preference distribution also could not show high distribution characteristics at both ends and low distribution characteristics in the middle. More than half of the network units prefer the symbolic numbers 0 and 5. The average goodness-of-fit of the Gaussian fitting of tuning curves increases with the increase in abscissa non-linearity. These results revealed that the concept of human symbolic number is trained on the basis of number sense.

Wavelet-based neural network with genetic algorithm optimization for generation prediction of PV plants

In order to solve the problem of uncertainty of photovoltaic power generation forecast, the prediction accuracy of photovoltaic power station power generation is further improved. In this paper, a wavelet neural network (WNN) based genetic algorithm (GA) is proposed to optimize the output forecasting method of photovoltaic power plants. It adopts the wavelet basis function as the transfer function of the network, and regards the network connection weight, the scaling factor and the translation factor of the wavelet function as a genetic individual. Then, it obtains the optimal initial parameters of the network through individual optimization based on genetic algorithm, and then imports it into the network. The results show that: under four typical weather types, the experimental data measured by photovoltaic power plants every half an hour are imported into four types of back-propagation artificial neural network (BP-ANN), WNN, GA-BP and GA-WNN network for simulation and prediction. The GA-WNN network prediction model outperforms the other three prediction methods in considering prediction error, relative error, other error evaluation methods and prediction reliability. The GA-WNN network is more suitable for the power generation forecast of photovoltaic power plants.

An Assessment and Analysis Model of Psychological Health of College Students Based on Convolutional Neural Networks.

Psychological health assessment and psychological problem identification essentially belong to problems of pattern recognition or nonlinear classification; its system contains complex nonlinear interactions among various factors, having basic characteristics of multivariable, multilevel, and strong coupling. An important problem in the field of artificial intelligence solved by convolutional neural networks (CNN) is to simplify complex problems, minimize the number of parameters, and thus greatly improve the algorithm's performance. Therefore, CNN has outstanding advantages in establishing the assessment and analysis model of college students' psychological health. This study determined the psychological health standards of college students, selected measurement tools for college students' psychological state, elaborated the principles of psychological assessment based on text information, performed the sample set data establishment and data processing of the assessment and analysis model of psychological health, conducted network establishment, training, and simulation, carried out a case experiment and its result analysis, explored the cause analysis of college students' psychological health problems, and finally discussed the prevention and intervention of college students' psychological problems. The study results show that the input and output of the CNN-based assessment and analysis model of college students' psychological health are their evaluation data and assessment results, respectively, and the optimal hyperparameters of the model are determined through fold cross-validation analysis to improve the model's over-fitting problem. After the training is completed, the model can predict the changes in college students' psychological state in the future through the psychological test data. The CNN uses supervised machine learning method to construct an assessment and analysis model of college students' psychological health, and establishes the mapping relationship between college students' personal background and their psychological health. The network error continuously adjusts network connection weight according to gradient descent algorithm to minimize its error, so that the convolutional layer and the pooling layer can learn the optimized feature expression of the input data.

Role of Lateral Inhibition on Visual Number Sense.

Newborn animals, such as 4-month-old infants, 4-day-old chicks, and 1-day-old guppies, exhibit sensitivity to an approximate number of items in the visual array. These findings are often interpreted as evidence for an innate “number sense.” However, number sense is typically investigated using explicit behavioral tasks, which require a form of calibration (e.g., habituation or reward-based training) in experimental studies. Therefore, the generation of number sense may be the result of calibration. We built a number-sense neural network model on the basis of lateral inhibition to explore whether animals demonstrate an innate “number sense” and determine important factors affecting this competence. The proposed model can reproduce size and distance effects of output responses of number-selective neurons when network connection weights are set randomly without an adjustment. Results showed that number sense can be produced under the influence of lateral inhibition, which is one of the fundamental mechanisms of the nervous system, and independent of learning.

Compression and Storage Algorithm of Key Information of Communication Data Based on Backpropagation Neural Network

This paper presents a backpropagation neural network algorithm for data compression and data storage. Data compression or establishing model ten coding is the most basic idea of traditional data compression. The traditionally designed ideas are mainly based on reducing the redundancy in the information and focus on the coding design, and its compression ratio has been hovering around dozens of percent. After the traditional coding compression of information, it is difficult to further compress by similar methods. In order to solve the above problems, the information that takes up less signal space can be used to represent the information that takes up more signal space to realize data compression. This new design idea of data compression breaks through the traditional limitation of relying only on coding to reduce data redundancy and achieves a higher compression ratio. At the same time, the information after such compression can be repeatedly compressed, and it has a very good performance. This is the basic idea of the combination of neural network and data compression introduced in this paper. According to the theory of multiobjective function optimization, this paper puts forward the theoretical model of multiobjective optimization neural network and studies a multiobjective data compression method based on neural network. According to the change of data characteristics, this method automatically adjusts the structural parameters (connection weight and bias value) of neural network to obtain the largest amount of data compression at the cost of small information loss. This method has the characteristics of strong adaptability, parallel processing, knowledge distributed storage, and anti-interference. Experimental results show that, compared with other methods, the proposed method has significant advantages in performance index, compression time and compression effect, high efficiency. and high-quality robustness.

Construction of the rat brain spatial cell firing model on a quadruped robot

Physiological studies have shown that rats in a dark environment rely on the limbs and vestibule for their self-motion information, which can maintain the specific firing patterns of grid cells and hippocampal CA3 place cells. In the development stage of rats, grid cells are considered to come from place cells, and place cells can be encoded by hippocampal theta cells. Based on these, the quadruped robot is used as a platform in this paper. Firstly, the sensing information of the robot's limbs and inertial measurement unit is obtained to solve its position in the environment. Then the position information is encoded by theta cells and mapped to place cells through a neural network. After obtaining the place cells with single-peak firing fields, Hebb learning is used to adjust the connection weight of the neural network between place cells and grid cells. In order to verify the model, 3-D simulation experiments are designed in this paper. The experiment results show that with the robot exploring in space, the spatial cells firing effects obtained by the model are consistent with the physiological research facts, which lay the foundation for the bionic environmental cognition model.

The Performance Improvement of VLC-OFDM System Based on Reservoir Computing

Nonlinear effects have been restricting the development of high-speed visible light communication (VLC) systems. Neural network (NN) has become an effective means to alleviate the nonlinearity of a VLC system due to its powerful ability to fit complicated functions. However, the complex training process of traditional NN limits its application in high-speed VLC. Without performance penalty, reservoir computing (RC) simplifies the training process of NN by training only part of the network connection weights, and has become an alternative scheme to NN. For the indoor visible light orthogonal frequency division multiplexing (VLC-OFDM) system, this paper studies the signal recovery effect of the pilot-assisted reservoir computing (PA-RC) frequency domain equalization algorithm. The pilot information is added to the feature engineering of RC to improve the accuracy of channel estimation by traditional least squares (LS) algorithm. The performance of 64 quadrature amplitude modulation (QAM) signal under different transmission rates and peak to peak voltage (Vpp) conditions is demonstrated in the experiments. Compared with the traditional frequency domain equalization algorithms, PA-RC can further expand the Vpp range that meets the 7% hard-decision forward error correction (FEC) limit of 3.8 × 10−3. At the rate of 240 Mbps, the BER of the system is reduced by about 90%, and the utilization rate of the available frequency band of the system reaches 100%. The results show that PA-RC can effectively improve the transmission performance of VLC system well, and has strong generalization ability.

A new adaptive sliding mode controller based on the RBF neural network for an electro-hydraulic servo system

Accuracy and robust trajectory tracking for electro-hydraulic servo systems in the presence of load disturbances and model uncertainties are of great importance in many fields. In this work, a new adaptive sliding mode control method based on the RBF neural networks (SMC–RBF) is proposed to improve the performances of a robotic excavator. Model uncertainties and load disturbances of the electro-hydraulic servo system are approximated and compensated using the RBF neural networks. Adaptive mechanisms are designed to adjust the connection weights of the RBF neural networks in real time to guarantee the stability. A nonlinear term is introduced into the sliding mode to design an adaptive terminal sliding mode control structure to improve dynamic performances and the convergence speed. Moreover, a sliding mode chattering reduction method is proposed to suppress the chattering phenomenon. Three types of step, ramp and sine signals are used as the simulation reference trajectories to compare different controllers on a co-simulation platform. Experiments with leveling and triangle conditions are presented on a robotic excavator. Results show that the proposed SMC–RBF controller is superior to existing proportional integral derivative (PID) and sliding mode controller (SMC) in terms of tracking accuracy and disturbance rejection.

Robust design of compact axial compressor

The method of connection weights in neural networks was used to analyze the sensitivity of the compressor rotor, and the Back Propagation Neural Network (BPNN) was used to construct the analysis relationship between the compressor rotor's geometries and the performance based on the training and learning of the data base, and the prediction accuracy can reach more than 99.99%. Then the modified Grason Algorithm based on the neural network connect weights was used to quantify the contribution of the geometrical effects on its performance. The result shows that the tip clearance contributes 11.43% (efficiency sensitivity analysis) and 10.18% (pressure ratio sensitivity analysis) to compressor performance changes. This study focuses mainly on the robust optimization of tip clearance. Non-intrusive probability collection point method (NIPC) was adopted for the uncertainty propagation. The robust optimization method based on BPNN agent model coupled with multi-objective genetic algorithm Non-dominated sorting genetic algorithm-II (NSGA II) was used to perform the optimization. Compared to the design prototype, the variance of robust compressor rotor's efficiency could be reduced by 21.04%.