Multilayer Perceptron Architecture Research Articles

Performance evaluation of artificial neural networks in estimating reference evapotranspiration with minimal meteorological data

Detailed meteorological data required for the equation of FAO-56 Penman-Monteith (P-M) method that was adopted by Food and Agriculture Organization (FAO) as a standard method in estimating reference evapotranspiration (ETo) are not often available, especially in developing countries. The Hargreaves equation (HG) has been successfully used in some locations to estimate ETo where sufficient data were not available to use the P-M method. This paper investigates the potential of two Artificial Neural Network (ANN) architectures, the multilayer perceptron architecture, in which a backpropagation algorithm (BPANN) is used, and the cascade correlation architecture (CCANN), in which Kalman’s learning rule is embedded in modeling the daily ETo with minimal meteorological data. An overview of the features of ANNs and traditional methods such as P-M and HG is presented, and the advantages and limitations of each method are discussed. Daily meteorological data from three automatic weather stations located in Greece were used to optimize and test the different models. The exponent value of the HG equation was locally optimized, and an adjusted HGadj equation was used. The comparisons were based on error statistical techniques using P-M daily ETo values as reference. According to the results obtained, it was found that taking into account only the mean, maximum and minimum air temperatures, the selected ANN models markedly improved the daily ETo estimates and provided unbiased predictions and systematically better accuracy compared with the HGadj equation. The results also show that the CCANN model performed better than the BPANN model at all stations.

Artificial neural networks predictive models. A case study: carbon and bromine concentrations prediction based on chlorination time

Artificial neural networks (ANNs) are being used increasingly to predict water variables. This study offers an alternative approach to quantify the relationship between time of chlorination in potable water (due to convectional treatment procedure) and chlorination by-products concentration (expressed as carbon and bromine) with an ANN model, i.e., capturing non-linear relationships among the water quality variables. Thus, carbon and bromine concentrations in potable water (the second chosen due to the toxicity of brominated trihalomethanes, THMs) were predicted using artificial neural networks (ANNs) based mainly on multi-layer perceptrons (MLPs) architecture. The chlorination (detention) time as much as 58 hours in Athens distributed network, comprised the input variables to the ANNs models. Moreover, to develop an ANN model for estimating carbon and bromine, the available data set was partitioned into training, validation and test set. In order to reach an optimum amount of hidden layers or nodes, different architectures were tested. The quality of the ANN simulations was evaluated in terms of the error in the validation sample set for the proper interpretation of the results. The calculated sum-squared errors for training, validation and test set were 0.056, 0.039 and 0.060 respectively for the best model selected. Comparison of the results showed that a two-layer feed-forward back propagation ANN model could be used as an acceptable model for predicting carbon and bromine contained in potable water THMs.

Support Vector Machine GARCH and Neural Network GARCH Models in Modeling Conditional Volatility: An Application to Turkish Financial Markets

The Turkish version of this paper can be found at: http://ssrn.com/abstract=2222071 The study aims to investigate linear GARCH, fractionally integrated FI-GARCH and Asymmetric Power APGARCH models and their nonlinear counterparts based on Support Vector Regression (SVR) and Neural Network (NN) models. GARCH family models are extended to NN-GARCH architecture of Donaldson and Kamstra (1997) to various NN-GARCH family models (Bildirici and Ersin, 2009) such as NN-APGARCH model. The study aims to introduce a class of extended NN-GARCH and SVR-GARCH family of models with nonlinear augmentations in modeling both the conditional mean and variance. The SVR-GARCH, SVR-APGARCH and SVR-FIAPGARCH and their Multi-Layer Perceptron architecture based counterparts, MLP-GARCH, MLP-APGARCH and MLP-FIAPGARCH are evaluated. An application to daily returns in Istanbul ISE100 stock index is provided. Results suggest that volatility clustering, asymmetry and nonlinearity characteristics are modeled more efficiently with the models possessing neural network architectures.

Neural network stochastic simulation applied for quantifying uncertainties

Generally the geostatistical simulation methods are used to generate several realizations of physical properties in the sub-surface, these methods are based on the variogram analysis and limited to measures correlation between variables at two locations only. In this paper, we propose a simulation of properties based on supervised Neural network training at the existing drilling data set. The major advantage is that this method does not require a preliminary geostatistical study and takes into account several points. As a result, the geological information and the diverse geophysical data can be combined easily. To do this, we used a neural network with multi-layer perceptron architecture like feed-forward, then we used the back-propagation algorithm with conjugate gradient technique to minimize the error of the network output. The learning process can create links between different variables, this relationship can be used for interpolation of the properties on the one hand, or to generate several possible distribution of physical properties on the other hand, changing at each time and a random value of the input neurons, which was kept constant until the period of learning. This method was tested on real data to simulate multiple realizations of the density and the magnetic susceptibility in three-dimensions at the mining camp of Val d'Or, Québec (Canada).

Digital Implementation of Artificial Neural Network for Function Approximation and Pressure Control Applications

The soft computing algorithms are being nowadays used for various multi input multi output complicated non linear control applications. This paper presented the development and implementation of back propagation of multilayer perceptron architecture developed in FPGA using VHDL. The usage of the FPGA (Field Programmable Gate Array) for neural network implementation provides flexibility in programmable systems. For the neural network based instrument prototype in real time application. The conventional specific VLSI neural chip design suffers the limitation in time and cost. With low precision artificial neural network design, FPGA have higher speed and smaller size for real time application than the VLSI design. The challenges are finding an architecture that minimizes the hardware cost, maximizing the performance, accuracy. The goal of this work is to realize the hardware implementation of neural network using FPGA. Digital system architecture is presented using Very High Speed Integrated Circuits Hardware Description Language (VHDL)and is implemented in FPGA chip. MATLAB ANN programming and tools are used for training the ANN. The trained weights are stored in different RAM, and is implemented in FPGA. The design was tested on a FPGA demo board. Keywords- Backpropagation, field programmable gate array (FPGA) hardware implementation, multilayer perceptron, pressure sensor, Xilinx FPGA.

Artificial Neural Networks for Characterizing Whipple Shield Performance

An artificial neural network model has been developed for predicting the perforation limits of spaced aluminum armor (aka Whipple shield) under impact of aluminum projectiles at hypervelocity. The network, utilizing a multilayer perceptron architecture, was trained on data from 769 impact tests, for which it accurately predicted the perforation of the shield rear wall (or lack thereof) 92% of the time. Comparatively, the leading empirical approach is capable of accurately predicting the outcome of 71% of the impact tests. The network output is an analogue probability of perforation, which is adjusted to a binary “Pass”/“Fail” result via a sigmoid fit that also provides physically plausible confidence bounds (i.e. error bars). Interrogation of the network was performed to identify the input parameters that most heavily correlated with the output prediction. Although the majority of the traditional parameters (i.e. those identified in the empirical ballistic limit equation) were amongst the most influential, some unexpected (and potentially spurious) parameters were also identified. A more widely sampled set of training data incorporating increased diversity in projectile and target materials would likely improve the network internal weighting for material properties and avoid accidental identification of biases in the training exemplars.

Artificial neural networks for characterising Whipple shield performance

An artificial neural network model has been developed for predicting the perforation limits of spaced aluminium armour (aka Whipple shield) under impact of aluminium projectiles at hypervelocity. The network, utilizing a multilayer perceptron architecture, was trained on data from 769 impact tests, for which it accurately predicted the perforation of the shield rear wall (or lack thereof) 92% of the time. Comparatively, the leading empirical approach was found to accurately predict the outcome of 71% of the impact tests. The network output is an analogue probability of perforation, which is adjusted to a binary “Pass”/”Fail” result via a sigmoid fit that also provides physically plausible confidence bounds (i.e. error bars). Interrogation of the network was performed to identify the input parameters that most heavily correlated with the output prediction. Although the majority of the traditional parameters (i.e. those identified in the empirical ballistic limit equation) were amongst the most influential, some unexpected (and potentially spurious) parameters were also identified. A more widely sampled set of training data incorporating increased diversity in projectile and target materials would likely improve the network internal weighting for material properties and avoid accidental identification of biases in the training exemplars.

Multilayer Perceptron Classification of Unknown Volatile Chemicals from the Firing Rates of Insect Olfactory Sensory Neurons and Its Application to Biosensor Design

In this letter, we use the firing rates from an array of olfactory sensory neurons (OSNs) of the fruit fly, Drosophila melanogaster, to train an artificial neural network (ANN) to distinguish different chemical classes of volatile odorants. Bootstrapping is implemented for the optimized networks, providing an accurate estimate of a network's predicted values. Initially a simple linear predictor was used to assess the complexity of the data and was found to provide low prediction performance. A nonlinear ANN in the form of a single multilayer perceptron (MLP) was also used, providing a significant increase in prediction performance. The effect of the number of hidden layers and hidden neurons of the MLP was investigated and found to be effective in enhancing network performance with both a single and a double hidden layer investigated separately. A hybrid array of MLPs was investigated and compared against the single MLP architecture. The hybrid MLPs were found to classify all vectors of the validation set, presenting the highest degree of prediction accuracy. Adjustment of the number of hidden neurons was investigated, providing further performance gain. In addition, noise injection was investigated, proving successful for certain network designs. It was found that the best-performing MLP was that of the double-hidden-layer hybrid MLP network without the use of noise injection. Furthermore, the level of performance was examined when different numbers of OSNs used were varied from the maximum of 24 to only 5 OSNs. Finally, the ideal OSNs were identified that optimized network performance. The results obtained from this study provide strong evidence of the usefulness of ANNs in the field of olfaction for the future realization of a signal processing back end for an artificial olfactory biosensor.

A Neural Network Based Hybrid Mixture Model to Extract Information from Non-linear Mixed Pixels

Signals acquired by sensors in the real world are non-linear combinations, requiring non-linear mixture models to describe the resultant mixture spectra for the endmember’s (pure pixel’s) distribution. This communication discusses inferring class fraction through a novel hybrid mixture model (HMM). HMM is a three-step process, where the endmembers are first derived from the images themselves using the N-FINDR algorithm. These endmembers are used by the linear mixture model (LMM) in the second step that provides an abundance estimation in a linear fashion. Finally, the abundance values along with the training samples representing the actual ground proportions are fed into neural network based multi-layer perceptron (MLP) architecture as input to train the neurons. The neural output further refines the abundance estimates to account for the non-linear nature of the mixing classes of interest. HMM is first implemented and validated on simulated hyper spectral data of 200 bands and subsequently on real time MODIS data with a spatial resolution of 250 m. The results on computer simulated data show that the method gives acceptable results for unmixing pixels with an overall RMSE of 0.0089 ± 0.0022 with LMM and 0.0030 ± 0.0001 with the HMM when compared to actual class proportions. The unmixed MODIS images showed overall RMSE with HMM as 0.0191 ± 0.022 as compared to the LMM output considered alone that had an overall RMSE of 0.2005 ± 0.41, indicating that individual class abundances obtained from HMM are very close to the real observations.

Classifying patterns with missing values using Multi-Task Learning perceptrons

Datasets with missing values are frequent in real-world classification problems. It seems obvious that imputation of missing values can be considered as a series of secondary tasks, while classification is the main purpose of any machine dealing with these datasets. Consequently, Multi-Task Learning (MTL) schemes offer an interesting alternative approach to solve missing data problems. In this paper, we propose an MTL-based method for training and operating a modified Multi-Layer Perceptron (MLP) architecture to work in incomplete data contexts. The proposed approach achieves a balance between both classification and imputation by exploiting the advantages of MTL. Extensive experimental comparisons with well-known imputation algorithms show that this approach provides excellent results. The method is never worse than the traditional algorithms – an important robustness property – and, also, it clearly outperforms them in several problems.

Markov Switching Artificial Neural Networks and Volatility Modeling with an Application to a Turkish Stock Index

The study analyzes the family of regime switching GARCH neural network models, which allow the generalization of MS type RS-GARCH models to MS-GARCH-NN models by incorparating with neural network architectures with different dynamics and forecasting capabilities both in addition to the family of GARCH models. In addition to the Gray (1996) RS-GARCH model which allows for within regime heteroskedasticity with markov switching of Hamilton (1989), the models analyzed in the study allow regime switching modeled with GARCH-NN specifications developed by Donaldson and Kamstra (1996) and further investigated by Bildirici and Ersin (2009). In addition to regime swiching type nonlinearity, proposed models incorporate different neural network architectures based on Multi Layer Perceptron (MLP), and Hybrid MLP models. Obtained models are MS-GARCH-MLP and MS-GARCH-Hybrid MLP. Above mentioned models are further extended to account for fractional integration (FI) in GARCH specification to obtain MS-FIGARCH-MLP and MS-FIGARCH-Hybrid MLP. By allowing asymmetric power transformation as modeled in APGARCH model, models are augmented to obtain MS-APGARCH-RBF and MS-FIGARCH-Hybrid MLP. Models are evaluated with MAE, MSE and RMSE criteria and equal forecast accuracy is tested with modified Diebold-Mariano tests. Among the models analyzed, though models which allow fractional integration and asymmetric power transformation perform better in modeling the daily returns in IMKB100 stock index, hybrid MLP and time lag recurrent architectures such as MS-FIAPGARCH-HybridMLP provide significant forecast and modeling performance. Overall, results suggest models with markov switching and neural network methodologies in modeling volatility in forecasting future returns in an emerging market stock index.

Modeling Markov Switching ARMA-GARCH Neural Networks Models and an Application to Forecasting Stock Returns

The study proposes and a family of regime switching GARCH neural network models to model volatility. The proposed MS-ARMA-GARCH-NN models allow MS type regime switching in both the conditional mean and conditional variance for time series and further augmented with artificial neural networks to achieve improvement in forecasting capabilities. Gray (1996) RS-GARCH model allows within regime heteroskedasticity with markov switching of Hamilton (1989). Firstly, models are extended to fractional integration and asymmetric power GARCH and MS-ARMA-FIGARCH, MS-ARMA-APGARCH, MS-ARMA-FIAPGARCH models are evaluated and discussed. Secondly, in addition to regime swiching type nonlinearity, neural networks based augmentated models are evaluated and discussed. In this respect, MS-ARMA-GARCH models are augmented with Multi Layer Perceptron (MLP), Radial Basis Function (RBF), Elman type Recurrent NN (Elman RNN), Time lag (delay) Recurrent NN (RNN) and Hybrid MLP models. Therefore, the second model group includes MS-ARMA-GARCH-MLP, MS-ARMA-GARCH-RBF, MS-ARMA-GARCH-ElmanRNN, MS-ARMA-GARCH-RNN and MS-ARMA-GARCH-HybridMLP. Thirdly, fractional integrated versions are augmented with neural networks and denoted as MS-ARMA-FIGARCH-MLP, MS-ARMA-FIGARCH-RBF, MS-ARMA-FIGARCH-ElmanRNN, MS-ARMA-FIGARCH-RNN and MS-ARMA-FIGARCH-HybridMLP. Fourth, by allowing asymmetric power transformations, models are further extended to GARCH models. These models are MS-ARMA-APGARCH-RBF, MS-ARMA-APGARCH-ElmanRNN, MS-ARMA-APGARCH-RNN and MS-ARMA-FIGARCH-HybridMLP. Therefore, a total of four group of GARCH models are proposed and evaluated in the study. In the empirical application section, daily stock returns in ISE100 Istanbul Stock Index are modeled and forecasted. Forecast success is evaluated with MAE, MSE and RMSE criteria. Equal forecast accuracy is tested with modified Diebold-Mariano tests. Accordingly, fractional integration and asymmetric power transformation perform better in modeling the daily returns in IMKB100 stock index compared to the simple GARCH and RS-GARCH model of Gray. Hybrid MLP and time lag recurrent architectures (MS-ARMA-FIAPGARCH-HybridMLP and MS-ARMA-FIAPGARCH-RNN) provided the best forecast and modeling performance. In conclusion, the newly proposed GARCH models have significant forecasting improvement and therefore are promising in various economic applications.

Application and performance analysis of neural networks for decision support in conceptual design

This paper analyzes the use of feedforward multilayer perceptron neural networks in the support of the decision process during the conceptual design phase of engineering systems. A user friendly software tool is proposed in order to increase the quality of the designers’ decisions by offering intuitive insights during the early design phase. The performance gain is evaluated through controlled experiments performed with non-experienced users.This paper proposes using the prediction capabilities of Artificial Neural Networks to recommend design solutions based on earlier successful designs. It implements a feedforward multilayer perceptron architecture with sigmoid activation functions, and uses the Levenberg–Marquardt training algorithm for weight matrix update. The network complexity was abstracted to ease the utilization by non-expert users. Different stopping conditions were developed including one where the training and testing error divergence is monitored to tackle the bias/variance dilemma.The advantages of this approach for decision support were measured through a set of six different case studies such as gaseous automobile emissions prediction based on institutional data, or the choice of a launcher for a specific space mission. The decision support tool generated quality performance gains between 13% and 88% for examples ranging from simple continuous single variable to complex discrete multivariate problems.

PREDICTION FOR TRAFFIC ACCIDENT SEVERITY: COMPARING THE ARTIFICIAL NEURAL NETWORK, GENETIC ALGORITHM, COMBINED GENETIC ALGORITHM AND PATTERN SEARCH METHODS

This paper focuses on predicting the severity of freeway traffic accidents by employing twelve accident-related parameters in a genetic algorithm (GA), pattern search and artificial neural network (ANN) modelling methods. The models were developed using the input parameters of driver's age and gender, the use of a seat belt, the type and safety of a vehicle, weather conditions, road surface, speed ratio, crash time, crash type, collision type and traffic flow. The models were constructed based on 1000 of crashes in total that occurred during 2007 on the Tehran–Ghom Freeway due to the fact that the remaining records were not suitable for this study. The GA evaluated eleven equations to obtain the best one. Then, GA and PS methods were combined using the best GA equation. The neural network used multi-layer perceptron (MLP) architecture that consisted of a multi-layer feed-forward network with hidden sigmoid and linear output neurons that could also fit multi-dimensional mapping problems arbitrarily well. The ANN was applied during training, testing and validation and had 12 inputs, 25 neurons in the hidden layers and 3 neurons in the output layer. The best-fit model was selected according to the R-value, root mean square errors (RMSE), mean absolute errors (MAE) and the sum of square error (SSE). The highest R-value was obtained for the ANN around 0.87, demonstrating that the ANN provided the best prediction. The combination of GA and PS methods allowed for various prediction rankings ranging from linear relationships to complex equations. The advantage of these models is improving themselves adding new data.

Comparing ANN to HMM in implementing limited Arabic vocabulary ASR systems

In this paper we investigated Artificial Neural Networks (ANN) based Automatic Speech Recognition (ASR) by using limited Arabic vocabulary corpora. These limited Arabic vocabulary subsets are digits and vowels carried by specific carrier words. In addition to this, Hidden Markov Model (HMM) based ASR systems are designed and compared to two ANN based systems, namely Multilayer Perceptron (MLP) and recurrent architectures, by using the same corpora. All systems are isolated word speech recognizers. The ANN based recognition system achieved 99.5% correct digit recognition. On the other hand, the HMM based recognition system achieved 98.1% correct digit recognition. With vowels carrier words, the MLP and recurrent ANN based recognition systems achieved 92.13% and 98.06, respectively, correct vowel recognition; but the HMM based recognition system achieved 91.6% correct vowel recognition.

Improving the Performance of Process Controllers Using a New Clustered Neural Network

This paper presents first a newly developed clustered neural network, which incorporates self-organization capacity into the well-known common multilayer perceptron (MLP) architecture. With this addition, it is possible to reduce significantly overall memory degradation of the neuro-controller during on-line training. In the second part of the paper, this clustered multilayer perceptron (CMLP) network is applied and compared to the MLP through modeling and simulations of machining processes. Simulation results presented using machining data demonstrate that the CMLP possesses more powerful modeling capacity than the standard MLP, offers better adaptability to new operating conditions, and finally performs more reliably. During on-line training with machining data about 65% degradation of previously learned information can be observed in the MLP as opposed to only 11% for the CMLP. Finally, an adaptive control scheme intended for on-line optimization of the machining processes is presented. This scheme uses a feed forward CMLP inverse neuro-controller which learns off-line and on-line the relationships between process inputs and output under simulated perturbations (i.e., tool wear and non-homogeneous workpiece material properties). The first results using the CMLP inverse neuro-controller are promising

Laparoscopy Pneumoperitoneum Fuzzy Modeling

AbstractAbstract: Gas volume to intra-peritoneal pressure fuzzy modeling for evaluating pneumoperitoneum in videolaparoscopic surgery is proposed in this paper. The proposed approach innovates in using fuzzy logic and fuzzy set theory for evaluating the accuracy of the prognosis value in order to minimize or avoid iatrogenic injuries due to the blind needle puncture. In so doing, it demonstrates the feasibility of fuzzy analysis to contribute to medicine and health care. Fuzzy systems is employed here in synergy with artificial neural network based on backpropaga tion, multilayer perceptron architecture for building up numerical functions. Experimental data employed for analysis were collected in the accomplishment of the pneumoperitoneum in a random population of patients submitted to videolaparoscopic surgeries. Numerical results indicate that the proposed fuzzy mapping for describing the relation from the intra peritoneal pressure measures as function injected gas volumes succeeded in determinining a fuzzy model for this nonlinear system when compared to the statistical model.

Development and Implementation of Parameterized FPGA-Based General Purpose Neural Networks for Online Applications

This paper presents the development and implementation of a generalized backpropagation multilayer perceptron (MLP) architecture described in VLSI hardware description language (VHDL). The development of hardware platforms has been complicated by the high hardware cost and quantity of the arithmetic operations required in online artificial neural networks (ANNs), i.e., general purpose ANNs with learning capability. Besides, there remains a dearth of hardware platforms for design space exploration, fast prototyping, and testing of these networks. Our general purpose architecture seeks to fill that gap and at the same time serve as a tool to gain a better understanding of issues unique to ANNs implemented in hardware, particularly using field programmable gate array (FPGA). The challenge is thus to find an architecture that minimizes hardware costs, while maximizing performance, accuracy, and parameterization. This work describes a platform that offers a high degree of parameterization, while maintaining generalized network design with performance comparable to other hardware-based MLP implementations. Application of the hardware implementation of ANN with backpropagation learning algorithm for a realistic application is also presented.

Downscaling Global Weather Forecast Outputs Using ANN for Flood Prediction

Downscaling global weather prediction model outputs to individual locations or local scales is a common practice for operational weather forecast in order to correct the model outputs at subgrid scales. This paper presents an empirical‐statistical downscaling method for precipitation prediction which uses a feed‐forward multilayer perceptron (MLP) neural network. The MLP architecture was optimized by considering physical bases that determine the circulation of atmospheric variables. Downscaled precipitation was then used as inputs to the super tank model (runoff model) for flood prediction. The case study was conducted for the Thu Bon River Basin, located in Central Vietnam. Study results showed that the precipitation predicted by MLP outperformed that directly obtained from model outputs or downscaled using multiple linear regression. Consequently, flood forecast based on the downscaled precipitation was very encouraging. It has demonstrated as a robust technology, simple to implement, reliable, and universal application for flood prediction through the combination of downscaling model and super tank model.

Constructing processing map of Ti40 alloy using artificial neural network

Based on the experimental data of Ti40 alloy obtained from Gleeble-1500 thermal simulator, an artificial neural network model of high temperature flow stress as a function of strain, strain rate and temperature was established. In the network model, the input parameters of the model are strain, logarithm strain rate and temperature while flow stress is the output parameter. Multilayer perceptron (MLP) architecture with back-propagation algorithm is utilized. The present study achieves a good performance of the artificial neural network (ANN) model, and the predicted results are in agreement with experimental values. A processing map of Ti40 alloy is obtained with the flow stress predicted by the trained neural network model. The processing map developed by ANN model can efficiently track dynamic recrystallization and flow localization regions of Ti40 alloy during deforming. Subsequently, the safe and instable domains of hot working of Ti40 alloy are identified and validated through microstructural investigations.