Traditional Multilayer Perceptron Research Articles

Evaluation of laser induced breakdown spectroscopy for multielemental determination in soils under sewage sludge application

Laser induced breakdown spectroscopy (LIBS) is an atomic emission spectroscopy technique for simple, direct and clean analysis, with great application potential in environmental sustainability studies. In a single LIBS spectrum it is possible to obtain qualitative information on the sample composition. However, quantitative analysis requires a reliable model for analytical calibration. Multilayer perceptron (MLP), an artificial neural network, is a multivariate technique that is capable of learning to recognize features from examples. Therefore MLP can be used as a calibration model for analytical determinations. Accordingly, the present study proposes to evaluate the traditional linear fit and MLP models for LIBS calibration, in order to attain a quantitative multielemental method for contaminant determination in soil under sewage sludge application. Two sets of samples, both composed of two kinds of soils were used for calibration and validation, respectively. The analyte concentrations in these samples, used as reference, were determined by a reference analytical method using inductively coupled plasma optical emission spectrometry (ICP OES). The LIBS–MLP was compared to a LIBS–linear fit method. The values determined by LIBS–MLP showed lower prediction errors, correlation above 98% with values determined by ICP OES, higher accuracy and precision, lower limits of detection and great application potential in the analysis of different kinds of soils.

Higher order and recurrent neural architectures for trading the EUR/USD exchange rate

The motivation for this paper is to investigate the use of alternative novel neural network architectures when applied to the task of forecasting and trading the Euro/Dollar (EUR/USD) exchange rate. This is done by benchmarking three different neural network designs representing a Higher Order Neural Network (HONN), a Psi Sigma Network and a Recurrent Network (RNN) with three successful architectures, the traditional Multilayer Perceptron (MLP), the Softmax and the Gaussian Mixture (GM) models. More specifically, the trading performance of the six models is investigated in a forecast and trading simulation competition on the EUR/USD time series over a period of 8 years. These results are also benchmarked with more traditional models such as a moving average convergence divergence technical model (MACD), an autoregressive moving average model (ARMA) and a logistic regression model (LOGIT). As it turns out, the MLP, the HONN, the Psi Sigma and the RNN models all do well and outperform the more traditional models in a simple trading simulation exercise. However, when more sophisticated trading strategies using confirmation filters and leverage are applied, the GM network produces remarkable results and outperforms all the other network architectures.

Knowledge-based evolving connectionist systems for condition evaluation of sustainable roadways: a feasibility study

In place, sustainable rehabilitation of existing deteriorated concrete highways through rubblization is considered to be a green and economical alternative to other options involving total reconstruction, etc. There is currently no standardised method of backcalculating the rubblized pavement layer moduli from pavement non-destructive test data through inverse analysis. This paper explores the feasibility of applying two evolving connectionist systems (ECOS), namely dynamic evolving neuro-fuzzy inference system (DENFIS) and evolving fuzzy neural network (EFuNN), to rubblized pavement moduli backcalculation. It is advantageous to employ ECOS networks for analysing complex engineered systems such as rubblized pavements since ECOS networks are resistant to catastrophic forgetting, have the ability to adapt to and learn new data as soon as they become available, do not have a limit to the amount of knowledge they can store and learn the examples very quickly compared to traditional multi-layered perceptron backpropagation neural networks (MLP-BP NN).

Pattern classification by multi-layer perceptron using fuzzy integral-based activation function

A multi-layer perceptron with single output node can be served as a classifier for two-class problems. Traditionally, an activation function such as the sigmoid function of a neuron performs the linear multi-regression model, which assumes that there is no interaction among attributes. However, because the interaction should not be ignored, this paper uses a non-linear fuzzy integral to replace the linear form by interpreting the connection weights as the values of the fuzzy measure and the degrees of importance of the respective input signals for the fuzzy integral-based sigmoid function. A fitness function of maximizing the number of correctly classified training patterns and minimizing the errors between the actual and desired outputs of individual training patterns is incorporated into the genetic algorithm to obtain appropriate parameter specifications. The experimental results further demonstrate that the perceptron with the fuzzy integral-based sigmoid function performs well in comparison with the traditional multi-layer perceptron.

Multiresolution-based bilinear recurrent neural network

A multiresolution-based bilinear recurrent neural network (MBLRNN) is proposed in this paper. The proposed MBLRNN is based on the BLRNN that has robust abilities in modeling and predicting time series. The learning process is further improved by using a multiresolution-based learning algorithm for training the BLRNN so as to make it more robust for the prediction of time series data. The proposed MBLRNN is applied to the problems of network traffic prediction and electric load forecasting. Experiments and results on both practical problems show that the proposed MBLRNN outperforms both the traditional multilayer perceptron type neural network (MLPNN) and the BLRNN in the prediction accuracy.

Incorporating a non-additive decision making method into multi-layer neural networks and its application to financial distress analysis

This paper presents a novel multi-layer perceptron using a non-additive decision making method and applies this model to the financial distress analysis, which is an important classification problem for a business, and the multi-layer perceptron has played a significant role in financial distress analysis. Traditionally, an activation function of an output neuron performs an additive method, namely the weighted sum method. Since the assumption of additivity among individual variables may not be reasonable, this paper uses a non-additive method, Choquet fuzzy integral, the fuzzy integral, to replace the weighted sum. In order to determine appropriate parameter specifications in the proposed model, a genetic algorithm is designed by considering the maximization of the number of correctly classified training patterns and the minimization of the training errors. The sample data obtained from Moody’s Industrial Manuals are employed to examine the classification ability of the proposed model. The results demonstrate that the proposed model performs well in comparison with the traditional multi-layer perceptron and some multivariate techniques.

A new approach for epileptic seizure detection using adaptive neural network

This paper presents new neural network models with adaptive activation function (NNAAF) to detect epileptic seizure. Our NNAAF models included three types named as NNAAF-1, NNAAF-2 and NNAAF-3. The activation function of hidden neuron in the model of NNAAF-1 is sigmoid function with free parameters. In the second model, NNAAF-2, activation function of hidden neuron is sum of sigmoid function with free parameters and sinusoidal function with free parameters. In the third model, NNAAF-3, hidden neurons' activation function is Morlet Wavelet function with free parameters. In addition, we implemented traditional multilayer perceptron (MLP) neural network (NN) model with fixed sigmoid activation function in the hidden layer to compare NNAAF models. The proposed models were trained and tested using 5-fold cross-validation to prove robustness of these models and to find the best model. We achieved 100% average sensitivity, average specificity, and approximately 100% average classification rate in all the models. It was seen that their speeds and the number of maximum iteration were changed for each model. The training time and the number of maximum iteration were reduced on about 50% using NNAAF-3 model. Hence it can be remarkable that NNAAF-3 is more suitable than the other models for real-time application.

Sparse solution in training artificial neural networks

Multilayer perceptrons (MLPs) for non-linear regression within a common framework with support vector machines (SVMs) and radial basis function regularised networks for non-linear regression are presented. The aim is taking advantage of the SVMs training paradigm to overcome the curse of dimensionality and too strict hypothesis on the statistics of errors in traditional MLPs for non-linear regression. In this context, an alternative strategy to quadratic programming, based on 1-norm minimisation to avoid computational problems of SVMs, is proposed.

Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.

Identification of discrete chaotic maps with singular points

We investigate the ability of artificial neural networks to reconstruct discrete chaotic maps with singular points. We use as a simple test model the Cusp map. We compare the traditional Multilayer Perceptron, the Chebyshev Neural Network and the Wavelet Neural Network. The numerical scheme for the accurate determination of a singular point is also developed. We show that combining a neural network with the numerical algorithm for the determination of the singular point we are able to accurately approximate discrete chaotic maps with singularities.

Identification and prediction of discrete chaotic maps applying a Chebyshev neural network

A new approach to reconstructing and predicting discrete chaotic maps is developed. It is based on the feed-forward neural network which decomposes the analyzed chaotic map in orthogonal Chebyshev polynomials. We show that the Chebyshev neural network (CNN) significantly exceeds the traditional multi-layer perceptron (MLP) in learning rate and in the accuracy of approximating an unknown map.

Identification of Chaotic Dynamics Using a Chebyshev Neural Network

A new approach to reconstructing discrete chaotic maps is developed. It is based on the feed-forward neural network which decomposes the analysed chaotic map in orthogonal Chebyshev polynomials. We show that the Chebyshev neural network (CNN) significantly exceeds the traditional multi-layer perceptron (MLP) in learning rate and in the accuracy of approximating an unknown map.

CMOS design of the Hao-Vandewalle model of neural associative memory

The design of the Hao–Vandewalle (1992) model of neural associative memories is presented. The model consists of three layers with neurons in the hidden layer and the output layer. There are weighted forward connections from the inputs to the hidden neurons and from the hidden neurons to the output neurons and there are no connections within the hidden layer or the output layer. This model utilizes nonlinear functions for the neurons in different layers and is different from the traditional three-layer Multilayer Perceptron (MLP). One important feature of this model is that it employs an extremely simple weight set-up rule and all the resulting weights can only assume two different values, −1 and +1, which facilitates VLSI implementation. A four-point multiplier was designed based on a very high frequency transconductor to function as a synapse. The designs of the hidden neuron and the output neuron for this model are also discussed. The implementation of a 5 × 4 CMOS Hao-Vandewalle Associative Memory is then presented.