Weights Of Artificial Neural Network Research Articles

Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications

Artificial Neural Networks (ANNs) are being used increasingly to predict and forecast water resources variables. In this paper, the steps that should be followed in the development of such models are outlined. These include the choice of performance criteria, the division and pre-processing of the available data, the determination of appropriate model inputs and network architecture, optimisation of the connection weights (training) and model validation. The options available to modellers at each of these steps are discussed and the issues that should be considered are highlighted. A review of 43 papers dealing with the use of neural network models for the prediction and forecasting of water resources variables is undertaken in terms of the modelling process adopted. In all but two of the papers reviewed, feedforward networks are used. The vast majority of these networks are trained using the backpropagation algorithm. Issues in relation to the optimal division of the available data, data pre-processing and the choice of appropriate model inputs are seldom considered. In addition, the process of choosing appropriate stopping criteria and optimising network geometry and internal network parameters is generally described poorly or carried out inadequately. All of the above factors can result in non-optimal model performance and an inability to draw meaningful comparisons between different models. Future research efforts should be directed towards the development of guidelines which assist with the development of ANN models and the choice of when ANNs should be used in preference to alternative approaches, the assessment of methods for extracting the knowledge that is contained in the connection weights of trained ANNs and the incorporation of uncertainty into ANN models.

Empirical comparison of various methods for training feed‐Forward neural networks for salinity forecasting

Feed‐forward artificial neural networks (ANNs) are being used increasingly to model water resources variables. In this technical note, six methods for optimizing the connection weights of feedforward ANNs are investigated in terms of generalization ability, parsimony, and training speed. These include the generalized delta (GD) rule, the normalized cumulative delta (NCD) rule, the delta‐bar‐delta (DBD) algorithm, the extended‐delta‐bar‐delta (EDBD) algorithm, the QuickProp (QP) algorithm, and the MaxProp (MP) algorithm. Each of these algorithms is applied to a particular case study, the forecasting of salinity in the River Murray at Murray Bridge, South Australia. Thirty models are developed for each algorithm, starting from different positions in weight space. The results obtained indicate that the generalization ability of the first‐order methods investigated (i.e., GD, NCD, DBD, and EDBD) is better than that of the second‐order algorithms (i.e., QP and MP). When the prediction errors are averaged over the 30 trials carried out, the performance of the first‐order methods in which the size of the steps taken in weight space is automatically adjusted in response to changes in the error surface (i.e., DBD and EDBD) is better than that obtained when predetermined step sizes are used (i.e., GD and NCD). However, the reverse applies when the best forecasts of the 30 trials are considered. The results obtained indicate that the EDBD algorithm is the most parsimonious and the MP algorithm is the least parsimonious. It was found that any impact different learning rules have on training speed is masked by the effect of epoch size and the number of hidden nodes required for optimal model performance.

Slowly Varying Nonstationary Traffic Flow Forecasting Using Bayesian Approach

The nonstationary traffic flow prediction is developed by using Bayesian approach in which the state conditioned upon the available measurement data is computed recursively. Both curvilinear regression approach and AR model for traffic flow forecasting are investigated. The parameters of above two approaches are adaptively updated by using Bayesian method. Adaptive neural networks (ATNN) is also adopted for the training procedure (Park et al., 1991a), which adapts the weights of a trained layered perceptron artificial neural network to training the data originated from a slowly varying nonstationary traffic flow processing. Through the simulation to traffic flow data got from a detector at a specific zone (#Z340) of Northern State Parkway (NY), the procedure is shown to adapt to new training data that is in conflict with earlier training data with least influence on neural network's response to previous data.

Making use of population information in evolutionary artificial neural networks

This paper is concerned with the simultaneous evolution of artificial neural network (ANN) architectures and weights. The current practice in evolving ANN's is to choose the best ANN in the last generation as the final result. This paper proposes a different approach to form the final result by combining all the individuals in the last generation in order to make best use of all the information contained in the whole population. This approach regards a population of ANN's as an ensemble and uses a combination method to integrate them. Although there has been some work on integrating ANN modules, little has been done in evolutionary learning to make best use of its population information. Four linear combination methods have been investigated in this paper to illustrate our ideas. Three real-world data sets have been used in our experimental studies, which show that the recursive least-square (RLS) algorithm always produces an integrated system that outperforms the best individual. The results confirm that a population contains more information than a single individual. Evolutionary learning should exploit such information to improve generalization of learned systems.

Towards designing artificial neural networks by evolution

Designing artificial neural networks (ANNs) for different applications has been a key issue in the ANN field. At present, ANN design still relies heavily on human experts who have sufficient knowledge about ANNs and the problem to be solved. As ANN complexity increases, designing ANNs manually becomes more difficult and unmanageable. Simulated evolution offers a promising approach to tackle this problem. This paper describes an evolutionary approach to design ANNs. The ANNs designed by the evolutionary process are referred to as evolutionary ANNs (EANNs). They represent a special class of ANNs in which evolution is another fundamental form of adaptation in addition to learning (also known as weight training). This paper describes an evolutionary programming (EP) based system to evolve both architectures and connection weights (including biases) of ANNs. Five mutation operators have been proposed in our evolutionary algorithm. In order to improve the generalisation ability of evolved ANNs, these five operators are applied sequentially and selectively. Validation sets have also been used in the evolutionary process in order to improve generalisation further. The evolutionary algorithm allows ANNs to grow as well as shrink during the evolutionary process. It incorporates the weight learning process as part of its mutation process. The whole EANN system can be regarded as a hybrid evolution and learning system. Extensive experimental studies have been carried out to test this EANN system. This paper gives some of the experimental results which show the effectiveness of the system.

Neural net-based monitoring of steel beams

An artificial neural network (ANN) approach was used to characterize vibration data for nondestructive evaluation purposes. Acoustic signatures were obtained from clamped–clamped metal beams of rectangular cross section. The beams were either intact, or had one (small) slot in them. The digitized data were used to train the ANN to predict future samples of the measured time series given past and present samples. The trained ANNs were used in two ways. In the first method, an ANN was trained with vibration data from intact beams. Once the ANN could adequately predict the training signal, vibration signals obtained from beams with slots were presented. Significant differences between prediction errors for the intact beam and beams with slots as shallow as 0.1 in. were found. Furthermore, the resulting prediction errors gradually increased as the slots in the beams grow deeper, suggesting that this method is useful to estimate defect size. In the second method, the connection weights of the ANNs trained on vibration data from intact beams were compared to the corresponding weights of ANNs retrained on the test data. Again, this approach was very sensitive, but no useful relationship with the slot depth was found.

An artificial neural network approach to quantitative single photon emission computed tomographic reconstruction with collimator, attenuation, and scatter compensation.

A spatially variant technique for quantitative single photon emission computed tomographic (SPECT) image reconstruction using an artificial neural network (ANN) is presented. This network was developed to simultaneously compensate for collimator, attenuation, and scatter effects during the reconstruction process. The network was trained using a supervised scheme which implemented the generalized delta rule. Training ended once the mean-squared error (MSE) between the ideal and reconstructed images converged to a minimum. After training, the ANN weights were held constant and could be used to reconstruct source distributions other than those used while training. In the absence of noise when only collimator effects were present, reconstruction of a Hoffman brain phantom had a 89% reduction in MSE compared to standard filtered backprojection. When collimator-and-attenuation and collimator-attenuation-and-scatter trials were tested against filtered backprojection with Chang attenuation compensation, the corresponding ANN reconstructions demonstrated 85% and 86% decreases in MSE, respectively. With noise present, and with standard noise reduction filters implemented prior to reconstruction, the ANN reconstructions displayed up to a 50% decrease in MSE compared to filtered backprojection reconstructions for 200,000 count data. These results demonstrate that an ANN can be used to reconstruct SPECT images with improved quantitative accuracy.

The Influence of Training Data Selection on Performance of Neural Networks for Control of Non-Linear Systems *

Abstract This work is part of research aimed at using artificial neural network models for real time process control over wide operating ranges where linear models either fail or must be adapted on-line. This paper discusses the influence on control performance of different methods for selecting and randomization/normalization of data used in estimating the weights in artificial neural network. The non-linear model system used comprises level control of a tank with non-vertical walls in which the level is controlled by manipulating out-flow and disturbances occures in the inflow. The simulation results show, that the control performance of the network is considerably influenced by the way in which the data for weight estimation are generated. A small randomized data set gives performance comparable to a data set sequential in time, which is many times larger. The performance of the trained artificial neural network in controlling the level of the tank is also compared with that of an IMC-PID controller. Results clearly demonstrate the advantage of the artificial neural network over the IMC-PID tuned at a nominal operating. The artificial neural network gives better performance over a wide operating range, because it accounts for the non-linear nature of the process.

The influence of training data selection on performance of neural networks for control of non-linear systems

This work is part of research aimed at using artificial neural network models for real time process control over wide operating ranges where linear models either fail or must be adapted on-line. This paper discusses the influence on control performance of different methods for selecting and randomization/normalization of data used in estimating the weights in artificial neural network. The non-linear model system used comprises level control of a tank with non-vertical walls in which the level is controlled by manipulating outflow and disturbances occures in the inflow. The simulation results show, that the control performance of the network is considerably influenced by the way in which the data for weight estimation are generated. A small randomized data set gives performance comparable to a data set sequential in time, which is many times larger. The performance of the trained artificial neural network in controlling the level of the tank is also compared with that of an IMC-PID controller. Results clearly demonstrate the advantage of the artificial neural network over the IMC-PID tuned at a nominal operating. The artificial neural network gives better performance over a wide operating range, because it accounts for the non-linear nature of the process.

ANALOG MOS VECTOR MULTIPLIERS FOR THE IMPLEMENTATION OF SYNAPSES IN ARTIFICIAL NEURAL NETWORKS

All-MOS analog vector-vector multipliers are described and designed for the implementation of the linear multiplication between analog signals and analog synaptic weights in artificial neural networks (ANNs). Employing these multipliers, large-scale artificial neural networks can be implemented using fewer MOS transistors than are required by implementations employing the so-called Gilbert multiplier. PSPICE circuit simulations have been extensively executed in order to quantify the performance of these multipliers by measuring the following specifications: maximum percentage error, output offset, X or Y nonlinearity, X or Y feedthrough, small-signal bandwidth, and slew rate. An 11-dimensional analog vector multiplier has been designed on a 40-pin MOSIS TINYCHIP with analog pads using the MAGIC VLSI tools and has been fabricated using 2 μm CMOS n-well process via MOSIS.

An adaptively trained neural network

A training procedure that adapts the weights of a trained layered perceptron artificial neural network to training data originating from a slowly varying nonstationary process is proposed. The resulting adaptively trained neural network (ATNN), based on nonlinear programming techniques, is shown to adapt to new training data that are in conflict with earlier training data without affecting the neural networks' response to data elsewhere. The adaptive training procedure also allows for new data to be weighted in terms of its significance. The adaptive algorithm is applied to the problem of electric load forecasting and is shown to outperform the conventionally trained layered perceptron.