Connection Weights Of Neural Network Research Articles

Optimizing connection weights in neural networks using the whale optimization algorithm

The learning process of artificial neural networks is considered as one of the most difficult challenges in machine learning and has attracted many researchers recently. The main difficulty of training a neural network is the nonlinear nature and the unknown best set of main controlling parameters (weights and biases). The main disadvantages of the conventional training algorithms are local optima stagnation and slow convergence speed. This makes stochastic optimization algorithm reliable alternative to alleviate these drawbacks. This work proposes a new training algorithm based on the recently proposed whale optimization algorithm (WOA). It has been proved that this algorithm is able to solve a wide range of optimization problems and outperform the current algorithms. This motivated our attempts to benchmark its performance in training feedforward neural networks. For the first time in the literature, a set of 20 datasets with different levels of difficulty are chosen to test the proposed WOA-based trainer. The results are verified by comparisons with back-propagation algorithm and six evolutionary techniques. The qualitative and quantitative results prove that the proposed trainer is able to outperform the current algorithms on the majority of datasets in terms of both local optima avoidance and convergence speed.

Application and Analysis of Fuzzy Neural Network Algorithm Based on Cloud Computing Platform in Enterprise Network Marketing

The emergence of cloud computing provides a convenient and efficient tool for solving the problem of enterprise network marketing. Cloud computing service mode can effectively reduce the cost of software and hardware, and flexibility to adapt to the needs of enterprises in different stages of development. Therefore, the construction mode of cloud computing will effectively solve the problem of enterprise network marketing. Combined with the characteristics of enterprise system and management, cloud computing can achieve data and application sharing between different devices with lower equipment requirements. With the characteristics of network marketing, this paper analyzes the basic structure of cloud computing, and proposes a kind of intuitionistic fuzzy neural network algorithm which is based on cloud computing. First, we divide the training sample into several sub blocks, and then put each sub block into the training network. After the training, we put the results into the membership network and non membership network. At the same time, we take the test sample to complete the calculation of the membership network and non membership network. Finally, the output value of the test sample and the training sample is synthesized by intuition, and the output value of the intuitionistic fuzzy neural network is obtained. In addition, we use MAP-REDUCE to determine the connection weights of neural networks in cloud computing. This effectively solves the problem of big data and long time consuming in the data mining network marketing information. Finally, we validate the algorithm on the cloud computing platform. The experimental results show that this paper has good application effect and prospect in the field of enterprise network marketing.

Reward-Modulated Hebbian Plasticity as Leverage for Partially Embodied Control in Compliant Robotics.

In embodied computation (or morphological computation), part of the complexity of motor control is offloaded to the body dynamics. We demonstrate that a simple Hebbian-like learning rule can be used to train systems with (partial) embodiment, and can be extended outside of the scope of traditional neural networks. To this end, we apply the learning rule to optimize the connection weights of recurrent neural networks with different topologies and for various tasks. We then apply this learning rule to a simulated compliant tensegrity robot by optimizing static feedback controllers that directly exploit the dynamics of the robot body. This leads to partially embodied controllers, i.e., hybrid controllers that naturally integrate the computations that are performed by the robot body into a neural network architecture. Our results demonstrate the universal applicability of reward-modulated Hebbian learning. Furthermore, they demonstrate the robustness of systems trained with the learning rule. This study strengthens our belief that compliant robots should or can be seen as computational units, instead of dumb hardware that needs a complex controller. This link between compliant robotics and neural networks is also the main reason for our search for simple universal learning rules for both neural networks and robotics.

Adaptive combination forecasting model for China’s logistics freight volume based on an improved PSO-BP neural network

Purpose – To accurately forecast logistics freight volume plays a vital part in rational planning formulation for a country. The purpose of this paper is to contribute to developing a novel combination forecasting model to predict China’s logistics freight volume, in which an improved PSO-BP neural network is proposed to determine the combination weights. Design/methodology/approach – Since BP neural network has the ability of learning, storing, and recalling information that given by individual forecasting models, it is effective in determining the combination weights of combination forecasting model. First, an improved PSO based on simulated annealing method and space-time adjustment strategy (SAPSO) is proposed to solve out the connection weights of BP neural network, which overcomes the problems of local optimum traps, low precision and poor convergence during BP neural network training process. Then, a novel combination forecast model based on SAPSO-BP neural network is established. Findings – Simulation tests prove that the proposed SAPSO has better convergence performance and more stability. At the same time, combination forecasting models based on three types of BP neural networks are developed, which rank as SAPSO-BP, PSO-BP and BP in accordance with mean absolute percentage error (MAPE) and convergent speed. Also the proposed combination model based on SAPSO-BP shows its superiority, compared with some other combination weight assignment methods. Originality/value – SAPSO-BP neural network is an original contribution to the combination weight assignment methods of combination forecasting model, which has better convergence performance and more stability.

Artificial Neural Network Learning Enhancement using Bacterial Foraging Optimization Algorithm

The artificial neural network (ANN) is a mathematical model capable of representing any non-linear relationship between input and output data. ANN is an abstract representation of the biological nervous system which has the ability to solve many complex problems. It has been successfully applied to a wide variety of classification and function approximation problems. The information processing capability of artificial neural networks (ANNs) is related to its architecture and weights. To have a high efficiency in ANN, selection of an appropriate architecture and learning algorithm is very important. In this study, the adaption of neural network connection weights using Bacterial Foraging Optimization Algorithm (BFO) is proposed as a mechanism to improve the performance of Artificial Neural Network in classification of Software Defect Dataset. The problem concerns the classification of software as defective or non defective on the basis of software metrics data. The results show that BFOANNs have better accuracy than traditional ANNs. The experimental results showed that BFOA-ANN has an improvement of 2.55 % in software defect prediction accuracy than the original feed forward artificial neural network and 2.80 % in case of cascade forward neural network. General Terms Learning enhancement, Optimization

Artificial Bee Colony training of neural networks: comparison with back-propagation

The Artificial Bee Colony (ABC) is a swarm intelligence algorithm for optimization that has previously been applied to the training of neural networks. This paper examines more carefully the performance of the ABC algo- rithm for optimizing the connection weights of feed-forward neural networks for classification tasks, and presents a more rigorous comparison with the traditional Back-Propagation (BP) training algorithm. The empirical results for bench- mark problems demonstrate that using the standard stop- ping early approach with optimized learning parameters leads to improved BP performance over the previous com- parative study, and that a simple variation of the ABC approach provides improved ABC performance too. With bothimprovementsapplied,theABCapproachdoesperform very well on small problems, but the generalization perfor- mances achieved are only significantly better than standard BP on one out of six datasets, and the training times increase rapidly as the size of the problem grows. If different, evo- lutionary optimized, BP learning rates are allowed for the two layers of the neural network, BP is significantly better than the ABC on two of the six datasets, and not significantly different on the other four.

Training neural networks with ant colony optimization algorithms for pattern classification

Feed-forward neural networks are commonly used for pattern classification. The classification accuracy of feed-forward neural networks depends on the configuration selected and the training process. Once the architecture of the network is decided, training algorithms, usually gradient descent techniques, are used to determine the connection weights of the feed-forward neural network. However, gradient descent techniques often get trapped in local optima of the search landscape. To address this issue, an ant colony optimization (ACO) algorithm is applied to train feed-forward neural networks for pattern classification in this paper. In addition, the ACO training algorithm is hybridized with gradient descent training. Both standalone and hybrid ACO training algorithms are evaluated on several benchmark pattern classification problems, and compared with other swarm intelligence, evolutionary and traditional training algorithms. The experimental results show the efficiency of the proposed ACO training algorithms for feed-forward neural networks for pattern classification.

Impulsive Neural Networks Algorithm Based on the Artificial Genome Model

To describe gene regulatory networks, this article takes the framework of the artificial genome model and proposes impulsive neural networks algorithm based on the artificial genome model. Firstly, the gene expression and the cell division tree are applied to generate spiking neurons with specific attributes, neural network structure, connection weights and specific learning rules of each neuron. Next, the gene segment duplications and divergence model are applied to design the evolutionary algorithm of impulsive neural networks at the level of the artificial genome. The dynamic changes of developmental gene regulatory networks are controlled during the whole evolutionary process. Finally, the behavior of collecting food for autonomous intelligent agent is simulated, which is driven by nerves. Experimental results demonstrate that the algorithm in this article has the evolutionary ability on large-scale impulsive neural networks

Radial Basis Function Neural Network Based on an Improved Exponential Decreasing Inertia Weight-Particle Swarm Optimization Algorithm for AQI Prediction

This paper proposed a novel radial basis function (RBF) neural network model optimized by exponential decreasing inertia weight particle swarm optimization (EDIW-PSO). Based on the inertia weight decreasing strategy, we propose a new Exponential Decreasing Inertia Weight (EDIW) to improve the PSO algorithm. We use the modified EDIW-PSO algorithm to determine the centers, widths, and connection weights of RBF neural network. To assess the performance of the proposed EDIW-PSO-RBF model, we choose the daily air quality index (AQI) of Xi’an for prediction and obtain improved results.

Based on Principal Components of Cooperative Co-evolutionary Genetic Neural Network in the Application of Stock Price Forecast

Considering that existing neural network’s weights are randomly selected; hidden layer nodes are difficult to determine; the convergence speed is slow, and all these will easily lead to the problems such as local minimum values. In order to solve these problems, this article puts forward a kind of cooperative co-evolutionary genetic neural network based on principal component. On the one hand, Cooperative Co-evolutionary Genetic Algorithm (CCGA) is introduced to optimize neural network structure and connection weights, so as to solve the problem that the hidden neurons are difficult to determine and avoid falling into the local minimum values; On the other hand, using Principal Component Analysis (PCA) to screen the input parameters of the network, so as to improve the convergence speed. The author established the model which is based on PCA-CCGA and so can simultaneously optimize the neural network’s structure and weights. The algorithm is proved to be effective through the simulation experiment of stock forecast.

Sensitivity Analysis in Artificial Neural Network and it's Applications on the Research of Attributes Correlation

Sensitivity analysis method can evaluate the importance of model input attributes. A multivariable sensitivity analysis method based on neural network connection weights and a calculation method of attributes correlation are proposed in this paper, and are applied to the research of attributes correlation. To verify the effectiveness of the proposed methods, this study employed a man-made example and a UCI-IRIS dataset to test the performance of the method. The results show that the sensitivity analysis method can really identify important and strong correlation attributes of model, and can simplify the model effectively, and can improve the accuracy of the model.

Research of the 3-DOF Helicopter System Based on Adaptive Inverse Control

For the flight control problem occurred in 3-DOF Helicopter System, reference adaptive inverse control scheme based on Fuzzy Neural Network model is designed. Firstly, fuzzy inference process of identifier and controller is achieved by using the network structure. Meanwhile, the neural network connection weights are used to express parameters of fuzzy inference. Then, back-propagation algorithm is adopted to amend the network connection weights in order to automatically identify the fuzzy model and adjust its membership functions and parameters, so that the actual system output of adaptive inverse controller control which is adjusted can track the reference model output. Finally, the simulation result of 3-DOF Helicopter System based on the scheme shows that the method is effective and feasible.

YIELD PREDICTION TECHNIQUE USING HYBRID ADAPTIVE NEURAL GENETIC NETWORK

The application of neural networks (NNs) to problems of prediction has become increasingly popular. This paper presents a modified hybrid adaptive neural network with revised adaptive smoothing errors, based on a genetic algorithm, and using modified adaptive relaxation to build a learning system for complex problem solving in yield prediction. This system predicts weekly yield values of a tomato crop using environmental variables measured inside the greenhouse as inputs. The proposed learning system is an intelligent computing technique and the numerical values of the neural network connection weights are modified through a training algorithm, using a modified optimization approach. The paper further presents an analysis of the convergence rate of the error in a neural network. The method is evaluated using datasets from a tomato producer, so as to test the predictive ability of the method and compare it with standard models. The results show a comparatively good level of accuracy.

A Neuro-Evolutionary Corpus-Based Method for Word Sense Disambiguation

The proposed approach to word sense disambiguation uses an evolutionary algorithm to automatically design the structure and learn the connection weights of neural networks.

Two algebraic criteria for input-to-state stability of recurrent neural networks with time-varying delays

This paper presents two algebraic criteria for the input-to-state stability of recurrent neural networks with time-varying delays. The criteria which also ensure global exponential stability when the input u(t) is equal to 0 and is easy to be verified only with the connection weights of the recurrent neural networks. Two numerical examples are given to demonstrate the effectiveness of the proposed criteria.

Application of Genetic Algorithm Optimized Neural Network Connection Weights for Medical Diagnosis of PIMA Indians Diabetes

Neural Networks are one of many data mining analytical tools that can be utilized to make predictions for medical data. Model selection for a neural network entails various factors such as selection of the optimal number of hidden nodes, selection of the relevant input variables and selection of optimal connection weights. This paper presents the application of hybrid model that integrates Genetic Algorithm and Back Propatation network(BPN) where GA is used to initialize and optmize the connection weights of BPN . Significant feactures identified by using two methods :Decision tree and GA-CFS method are used as input to the hybrid model to diagonise diabetes mellitus. The results prove that, GA-optimized BPN approach has outperformed the BPN approach without GA optimization. In addition the hybrid GA-BPN with relevant inputs lead to further improvised categorization accuracy compared to results produced by GA-BPN alone with some redundant inputs.

A genetic fuzzy radial basis function neural network for structural health monitoring of composite laminated beams

In this paper, a new neural network learning procedure, called genetic fuzzy hybrid learning algorithm (GFHLA) is proposed for training the radial basis function neural network (RBFNN). The method combines the genetic algorithm and fuzzy logic to optimize the centers and widths of the RBFNN, and the linear least-squared method is used to adjust the neural network connection weights. The modal frequencies of a glass/epoxy laminates beam with varying assumed delamination sizes and locations were computed using finite element method and fed into the genetic fuzzy RBF neural network to predict the delamination location and its extent. The simulation demonstrates that the neural network based on GFHLA is robust and promising.

Prediction of mine coal layer spontaneous combustion danger based on genetic algorithm and BP neural networks

The prediction of danger state in mine coal layer spontaneous combustion correctly has an important practical meaning for the mine production safety. There is a kind of complicated nonlinear relation between the danger of coal layer spontaneous combustion and its influencing factors. And the neural network can truly show the nonlinear relation. In this paper, for the purpose of predicting it in mine coal layer correctly, a kind of method combined the advantages of genetic algorithm (GA) and BP neural network is introduced based on the demonstration of the necessity and possibility of combining BP with GA. At first, the notion of using multilayered BP as the representation method of genetic and the searching technique is introduced, and a novel method of using GA to train connection weights of BP neural network is designed. According to the characteristics of mine coal layer spontaneous combustion danger, three key influencing factors are selected as the judging indexes. Then the model for predicting the danger of mine coal layer spontaneous combustion is built. Practical application indicates that the capability of the new method in fast learning of ANN and escaping local optima. And the results show that the model is a very efficient method for predicting the danger of coal layer spontaneous combustion.

Development of Neural-Net-Based Decision Support System for Mattress Patterns Using Particle Swarm Optimization

The particle swarm optimization (PSO) concept simulating a simplified social milieu, is optimization useful for solving nonconvex continuous optimization problems. We discuss a new learning algorithm for simultaneously adjusting connection weights in neural networks and user-specified parameters included in units. Based on our algorithm, neural network learning, e.g., learning cost or adaptability, can be improved, as demonstrated in mattress decision support system.

An intelligent computing technique for fluid flow problems using hybrid adaptive neural network and genetic algorithm

A new hybrid adaptive neural network (ANN) with modified adaptive smoothing errors (MASE) based on genetic algorithm (GA) employing modified adaptive relaxation (MAR) are presented in this paper to construct learning system for complex problem solving in fluid dynamics. This system can predict an incompressible viscous fluid flow represents by stream function ( ψ) through symmetrical backward-facing steps channels. The proposed learning system is constructed as an intelligent computing technique by enforcing three stages run simultaneously; the first stage concerns to construct finite-element method (FEM) employing a new approach named modified adaptive incremental loading (MAIL) to build-up in run-time a dataset driven that contains an effective patterns represented by ψ for specific Reynolds number ( Re), these patterns are associated to three kinds of clusters. The second stage is pertained a new hybrid neural network with new modification of adaptive smoothing errors and the third stage illustrated to modifying the numerical values of neural network connection weights through certain training algorithm with new optimization approach. The present simulation results of the proposed learning system are in good agreement with the available previous works and it is fast enough and stable.