Feedforward Multilayer Perceptron Research Articles

Unsupervised on-line method to diagnose unbalanced voltage in three-phase induction motor

In this paper, an unsupervised automatic method based on a current signature neural network (NN) is presented to on-line diagnose stator fault without the inspection of any supervisor or technician. To extract the fault regime, the knowledge of current signature will not be enough; therefore, mathematical model, numerical analysis, as well as artificial intelligence (AI) are taken into account to extract the exact unbalanced voltage stator fault. Analytical expressions are derived for a stator conductor segment in order to find out the conductors that are responsible for the generation of magnetomotive force (MMF). A test rig is designed using three-phase induction motor, two-axis PASPORT sensor, PC, and PASCO interface to compute the effect of MMF at the stator side through a new series of harmonics which are helpful to tackle the scrupulous effect of an unbalanced voltage at the incipient stage. Further, an unsupervised NN has been introduced that endeavors the principal components of the new series of harmonics. The statistical parameters of a new series of harmonics are contemplated as input features for NN that not only diagnose unbalanced voltage but also identify the degree of unbalanced voltage through feed-forward multilayer perceptron (MLP) trained by backpropagation. The validation and performance of proposed methods have been theoretically and experimentally analyzed on a custom-designed test rig under various levels of unbalanced voltage. Moreover, the NN classification method shows higher accuracy with enough robustness to various levels of unbalanced voltage, which states that the proposed method is suitable for the real-world applications.

Artificial neural network modeling for prediction of binary surface tension containing ionic liquid

ABSTRACTIn this study, a feed-forward multilayer perceptron neural network is applied to predict the surface tension of 32 binary ionic liquids (ILs)/non-ILs systems using melting point (Tm), molecular weight (Mw) and mole fraction of ILs as well as Tm and Mw of non-IL components. The data are divided into two different subsets, namely training and testing subsets, to obtain the optimum parameters of the used network and to evaluate the correlative capability of the trained network. The results of the test stage show excellent capability of the proposed network to predict/correlate the binary surface tension of ILs/non-ILs systems (AARD%: 0.93, MSE: 6.67 × 10−7 and R2: 0.9950).

Detecting redirection spam using multilayer perceptron neural network

Quality information retrieval from Web is essential for every search engine. But the quality of information is being exploited by spammers who make heavy use of malicious redirections for the purpose of phishing, downloading malware or attaining high search engine ranking. Malicious redirections present the irrelevant content to search user, thereby affecting user satisfaction. It also leads to wastage of network bandwidth. In this paper, we propose a neural framework for detecting redirection spam. We incorporated the feed-forward multilayer perceptron network and used scaled conjugate gradient algorithm that is able to perform very fast classification of URLs leading to redirection spam. We investigated the network empirically to choose the number of hidden layers and observed that when network is trained with two hidden layers, it gives better accuracy. We validated our proposed approach against the dataset of 2383 URLs and were able to detect the spammed redirections with high accuracy. The results indicate that neural networks are very effective technique to model the redirection spam detection.

Forecasting Foreign Exchange Rate Movements with k-Nearest-Neighbour, Ridge Regression and Feed-Forward Neural Networks

Three different classes of data mining methods (k-Nearest Neighbour, Ridge Regression and Multilayer Perceptron Feed-Forward Neural Networks) are applied for the purpose of quantitative trading on 10 simulated time series, as well as real world time series of 10 currency exchange rates ranging from 1.11.1999 to 12.6.2015. Each method is tested in multiple variants. The k-NN algorithm is applied alternatively with the Euclidian, Manhattan, Mahalanobis and Maximum distance function. The Ridge Regression is applied as Linear and Quadratic, and the Feed-Forward Neural Network is applied with either 1, 2 or 3 hidden layers. In addition to that Principal Component Analysis (PCA) is eventually applied for the dimensionality reduction of the predictor set and the meta-parameters of the methods are optimized on the validation sample. In the simulation study a Stochastic-Volatility Jump-Diffusion model, extended alternatively with 10 different non-linear conditional mean patterns, is used, to simulate the asset price behaviour to which the tested methods are applied. The results show that no single method was able to profit on all of the non-linear patterns in the simulated time series, but instead different methods worked well for different patterns. Alternatively, past price movements and past returns were used as predictors. In the case when the past price movements were used, quadratic ridge regression achieved the most robust results, followed by some of the k-NN methods. In the case when past returns were used, k-NN based methods were the most consistently profitable, followed by the linear ridge regression and quadratic ridge regression. Neural networks, while being able to profit on some of the time series, did not achieve profit on most of the others. No evidence was further found of the PCA method to improve the results of the tested methods in a systematic way. In the second part of the study, the models were applied to empirical foreign exchange rate time series. Overall the profitability of the methods was rather low, with most of them ending with a loss on most of the currencies. The most profitable currency was EURUSD, followed by EURJPY, GBPJPY and EURGBP. The most successful methods were the linear ridge regression and the Manhattan distance based k-NN method which both ended with profits for most of the time series (unlike the other methods). Finally, a forward selection procedure using the linear ridge regression was applied to extend the original predictor set with some technical indicators. The selection procedure achieved limited success in improving the out-sample results for the linear ridge regression model but not the other models.

Application of neural models as controllers in mobile robot velocity control loop

Abstract This paper presents the application of an inverse neural models used as controllers in comparison to classical PI controllers for velocity tracking control task used in two-wheel, differentially driven mobile robot. The PI controller synthesis is based on linear approximation of actuators with equivalent load. In order to obtain relevant datasets for training of feed-forward multi-layer perceptron based neural network used as neural model, the mathematical model of mobile robot, that combines its kinematic and dynamic properties such as chassis dimensions, center of gravity offset, friction and actuator parameters is used. Neural models are trained off-line to act as an inverse dynamics of DC motors with particular load using data collected in simulation experiment for motor input voltage step changes within bounded operating area. The performances of PI controllers versus inverse neural models in mobile robot internal velocity control loops are demonstrated and compared in simulation experiment of navigation control task for line segment motion in plane.

A method of parameterising a feed forward multi-layered perceptron artificial neural network, with reference to South African financial markets

No analytic procedures currently exist for determining optimal artificial neural network structures and parameters for any given application. Traditionally, when artificial neural networks have been applied to financial modelling problems, structure and parameter choices are often made a priori without sufficient consideration of the effect of such choices. A key aim of this study is to develop a general method that could be used to construct artificial neural networks by exploring the model structure and parameter space so that informed decisions could be made relating to the model design. In this study, a formal approach is followed to determine suitable structures and parameters for a Feed Forward Multi-layered Perceptron artificial neural network with a Resilient Propagation learning algorithm with a single hidden layer. This approach is demonstrated through the modelling of four South African economic variables, namely the average monthly returns on the money, bond and equity markets as well as monthly inflation. Artificial neural networks can be constructed on the aforementioned variables in isolation or, jointly, in an integrated model. The performance of a range of more traditional time series models is compared with that of the artificial neural network models. The results suggest that, on a statistical level, artificial neural networks perform as well as time series models at forecasting the returns for financial markets. Hybrid models, combining artificial neural networks with the time series models, are constructed, trained and tested for the money market and for the rate of inflation. They appear to add value to the time series models when forecasting inflation, but not for the money market.

Response surface methodology and artificial neural network modeling of reactive red 33 decolorization by O3/UV in a bubble column reactor

In this work, response surface methodology (RSM) and artificial neural network (ANN) were used to predict the decolorization efficiency of Reactive Red 33 (RR 33) by applying the O3/UV process in a bubble column reactor. The effects of four independent variables including time (20-60 min), superficial gas velocity (0.06-0.18 cm/s), initial concentration of dye (50-150 ppm), and pH (3-11) were investigated using a 3-level 4-factor central composite experimental design. This design was utilized to train a feed-forward multilayered perceptron artificial neural network with a back-propagation algorithm. A comparison between the models’ results and experimental data gave high correlation coefficients and showed that the two models were able to predict Reactive Red 33 removal by employing the O3/UV process. Considering the results of the yield of dye removal and the response surface-generated model, the optimum conditions for dye removal were found to be a retention time of 59.87 min, a superficial gas velocity of 0.18 cm/s, an initial concentration of 96.33 ppm, and a pH of 7.99.

Development of an ANN-based soft-sensor to estimate the apparent viscosity of water-based drilling fluids

Oil exploration in marine fields located in deepwater and ultra-deepwater has been one of the challenges in oil production. To enable exploration in such conditions, huge investments have been made, and extensive research has been carried out in order to improve technology to the drilling fluids. In this way, the understanding and the monitoring of the properties of the drilling fluids are of vital importance to the exploration and production of oil in deepwater and ultra-deepwater conditions. The aim of this work was to develop a soft-sensor based on an artificial neural network (ANN) to estimate the apparent viscosity of the water-based drilling fluids. In the initial phase of this work, the influence of the additives on the apparent viscosity of the drilling fluids was investigated. This procedure was carried out by means of a complete three-level factorial experiment design, with central point in triplicate. The additives (factors) used were the xanthan gum, bentonite and barite. Since the temperature of the drilling fluid that returns from the well is different than that in the injection point (the drill string) this variable was also considered in the factorial experimental design in order to evaluate its influence on the apparent viscosity. The artificial neural network used in this work was a feedforward multilayer perceptron (MLP) with hyperbolic activation functions in the hidden layer and linear activation function in the output neuron. It was found that the ANN model with six neurons in the hidden layer presented the best predictions. The performance of the selected ANN and the statistical regression models were compared each other in the sense of the mean squared error, and the neuron model showed better predictions of the apparent viscosity.

Automated detection of welding defects in pipelines from radiographic images DWDI

This paper presents a method for the automatic detection and classification of defects in radiographic images of welded joints obtained by exposure technique of double wall double image (DWDI). The proposed method locates the weld bead on the DWDI radiographic images, segments discontinuities (potential defects) in the detected weld bead and extracts features of these discontinuities. These features are used in a feed-forward multilayer perceptron (MLP) with backpropagation learning algorithm to classify descontinuities in “defect and no-defect”. The classifier reached an accuracy of 88.6% and a F-score of 87.5% for the test data. A comparison of the results with the earlier studies using SWSI and DWSI radiographic images indicates that the proposed method is promising. This work contributes towards the improvement of the automatic detection of welding defects in DWDI radiographic image which results can be used by weld inspectors as a support in the preparation of technical reports.

Spectrum occupancy prediction based on functional link artificial neural network (FLANN) in ISM band

Recent advancements in artificial neural networks (ANNs) motivated us to design a simple and faster spectrum prediction model termed the functional link artificial neural network (FLANN). The main objective of this paper is to gather realistic data to obtain utilization statistics for the industrial, scientific and medical band of 2.4–2.5 GHz. To present the occupancy statistics, we conducted measurement in indoors at the Swearingen Engineering Center, University of South Carolina. Further, we introduce different threshold-based spectrum prediction schemes to show the impact of threshold on occupancy, and propose a spectrum prediction algorithm based on FLANN to forecast a future spectrum usage profile from historical occupancy statistics. Spectrum occupancy is estimated and predicted by employing different ANN models including the Feed-forward multilayer perceptron (MLP), Recurrent MLP, Chebyshev FLANN and Trigonometric FLANN. It is observed that the absence of a hidden layer in FLANN makes it more efficient than the MLP model in predicting the occupancy faster and with less complexity. A set of illustrative results are presented to validate the performance of our proposed learning scheme.

Factors influencing the development of visceral metastasis of breast cancer: A retrospective multi-center study

PurposeVisceral metastasis of breast cancer (BC) is an alarming development and correlates with poor median overall survival. The purpose of this retrospective study is to examine the risk factors for developing visceral metastasis by considering tumor biology and patient characteristics. MethodsUsing the BRENDA database, the risk factors such as histological and intrinsic subtypes of BC, age at primary diagnosis, grading, nodal status, tumor size and year of primary diagnosis were examined in univariate and multivariate analysis. Categorical variables were compared by using χ2 tests. Furthermore, multivariate Cox proportional hazards regression models, Kaplan–Meier product-limit method and log-rank test were applied. The results of two tree-building algorithms, “exhausted CHAID” (Chi-squared Automatic Interaction Detector) and CART (Classification and Regression Trees) were verified with further multivariate analysis, radial basis function networks (RBF-net), feedforward multilayer perceptron networks (MLP) and logistic regression. ResultsIn a patient collective of 886 metastasized patients, 56.9% had developed visceral metastases and 27.1% visceral-only metastases. The different histological and intrinsic subtypes of BC and the grading correlate significantly with the visceral-only metastasis behavior, whereas the age at primary diagnosis, the nodal status, the tumor size and the year of the primary diagnosis had no influence. Patients with ductal/other BC, LuminalB/HER2, TNBC, HER2 overexpressing subtype and grade 3 had an increased risk for the development of visceral-only metastasis. ConclusionsIntrinsic and histological subtypes as well as the grading of BC affected significantly the visceral metastasis behavior.

Deep belief network based electricity load forecasting: An analysis of Macedonian case

A number of recent studies use deep belief networks (DBN) with a great success in various applications such as image classification and speech recognition. In this paper, a DBN made up from multiple layers of restricted Boltzmann machines is used for electricity load forecasting. The layer-by-layer unsupervised training procedure is followed by fine-tuning of the parameters by using a supervised back-propagation training method. Our DBN model was applied to short-term electricity load forecasting based on the Macedonian hourly electricity consumption data in the period 2008–2014. The obtained results are not only compared with the latest actual data, but furthermore, they are compared with the predicted data obtained from a typical feed-forward multi-layer perceptron neural network and with the forecasted data provided by the Macedonian system operator (MEPSO). The comparisons show that the applied model is not only suitable for hourly electricity load forecasting of the Macedonian electric power system, it actually provides superior results than the ones obtained using traditional methods. The mean absolute percentage error (MAPE) is reduced by up to 8.6% when using DBN, compared to the MEPSO data for the 24-h ahead forecasting, and the MAPE for daily peak forecasting is reduced by up to 21%.

Hydrological flow rate estimation using artificial neural networks: Model development and potential applications

In present paper we develop a model of monthly river flow rate using artificial neural networks, based on the assumption that air temperature and precipitation predetermine the flow rate dynamics. In order to create a reliable prediction model, we used monthly observations made at Lim river basin in southwestern Serbia from 1950 to 2012, since results of this analysis could be significant for hydro-energy production at ``Potpeć" and ``Brodarevo" hydropower plants. Analysis was conducted using multilayer feed-forward perceptron with Levenberg–Marquardt learning algorithm and appropriate number of hidden neurons which provide the most reliable prediction accuracy. Analysis of derived models with different number of hidden nodes indicate that models are insensitive to the number of hidden units. Model with eight hidden nodes was chosen as the most reliable one, providing highest prediction accuracy (the highest values of determination coefficient and Nash–Sutcliffe coefficient). Predictive power of the developed model was tested against the recordings made in period 1991–2012, providing satisfying prediction accuracy. Moreover, Monte-Carlo simulation showed that prediction accuracy of developed models is robust against expected experimental error, confirming that derived models provide reliable predictions of flow rates, which could be used for water management plans and strategies. We also propose two potential applications of derived model: for predicting the future flow rate using the predefined climate models, and for forecasting the hydroenergy production, on the basis of the linear dependence of the observed flow rate and previously produced electric power. These application are verified for the regional climate model EBU-POM, for the period 2013–2100, and using the data on electric power production at hydro powerplant ``Potpeć".

Evaluation of clinical parameters influencing the development of bone metastasis in breast cancer.

BackgroundThe development of metastases is a negative prognostic parameter for the clinical outcome of breast cancer. Bone constitutes the first site of distant metastases for many affected women. The purpose of this retrospective multicentre study was to evaluate if and how different variables such as primary tumour stage, biological and histological subtype, age at primary diagnosis, tumour size, the number of affected lymph nodes as well as grading influence the development of bone-only metastases.MethodsThis retrospective German multicentre study is based on the BRENDA collective and included 9625 patients with primary breast cancer recruited from 1992 to 2008. In this analysis, we investigated a subgroup of 226 patients with bone-only metastases. Association between bone-only relapse and clinico-pathological risk factors was assessed in multivariate models using the tree-building algorithms “exhausted CHAID (Chi-square Automatic Interaction Detectors)” and CART(Classification and Regression Tree), as well as radial basis function networks (RBF-net), feedforward multilayer perceptron networks (MLP) and logistic regression.ResultsMultivariate analysis demonstrated that breast cancer subtypes have the strongest influence on the development of bone-only metastases (χ2 = 28). 29.9 % of patients with luminal A or luminal B (ABC-patients) and 11.4 % with triple negative BC (TNBC) or HER2-overexpressing tumours had bone-only metastases (p < 0.001). Five different mathematical models confirmed this correlation. The second important risk factor is the age at primary diagnosis. Moreover, BC subcategories influence the overall survival from date of metastatic disease of patients with bone-only metastases. Patients with bone-only metastases and TNBC (p < 0.001; HR = 7.47 (95 % CI: 3.52–15.87) or HER2 overexpressing BC (p = 0.007; HR = 3.04 (95 % CI: 1.36–6.80) have the worst outcome compared to patients with luminal A or luminal B tumours and bone-only metastases.ConclusionThe bottom line of different mathematical models is the prior importance of subcategories of breast cancer and the age at primary diagnosis for the appearance of osseous metastases. The primary tumour stage, histological subtype, tumour size, the number of affected lymph nodes, grading and NPI seem to have only a minor influence on the development of bone-only metastases.

Neural-based prediction of structural failure of multistoried RC buildings

Various vague and unstructured problems encountered the civil engineering/designers that persuaded by their experiences. One of these problems is the structural failure of the reinforced concrete (RC) building determination. Typically, using the traditional Limit state method is time consuming and complex in designing structures that are optimized in terms of one/many parameters. Recent research has revealed the Artificial Neural Networks potentiality in solving various real life problems. Thus, the current work employed the Multilayer Perceptron Feed-Forward Network (MLP-FFN) classifier to tackle the problem of predicting structural failure of multistoried reinforced concrete buildings via detecting the failure possibility of the multistoried RC building structure in the future. In order to evaluate the proposed method performance, a database of 257 multistoried buildings RC structures has been constructed by professional engineers, from which 150 RC structures were used. From the structural design, fifteen features have been extracted, where nine features of them have been selected to perform the classification process. Various performance measures have been calculated to evaluate the proposed model. The experimental results established satisfactory performance of the proposed model.

Estimation of thermal conductivity of Al2O3/water (40%)–ethylene glycol (60%) by artificial neural network and correlation using experimental data

In this work, the estimation of thermal conductivity of Al2O3 nanoparticles in water (40%)–ethylene glycol (60%) has been investigated. An empirical relationship has been proposed based on experimental data and in terms of temperature and volume fraction. Besides, a model has been presented using feedforward multi-layer perceptron (MLP) artificial neural network (ANN). The presented correlation relationship estimates empirical data very well. However, artificial neural network has a higher regression coefficient and lower error compared to the presented relationship. After examining different structures of neural network with different transfer functions, a structure was selected with two hidden layers and 5 neurons in the first and second layers and tangent sigmoid transfer function for both layers. The results indicate that artificial neural networks can precisely estimate the experimental data of thermal conductivity of Al2O3/water (40%)–ethylene glycol (60%) nanofluids.

Intelligent Choice-Based Network Revenue Management

Choice-based network revenue management concentrates on importing choice models within the traditional revenue management system. Multinomial logit is a popular and well-known model which is base choice model in the revenue management. Empirical results indicate inadequacy of this model for predicting itinerary shares and more realistic models such as nested logit can be proposed for substituting it. Incorporating complex choice models in the optimization module based on statistical tests without considering the complexity of the obtained mathematical model, would lead to increase the complexity of a system without obtaining significant improvement. According to influencing the discrete choice model on the structure of optimization model, it is necessary to analyze the interaction between specific discrete choice and optimization models.In this paper, a knowledge acquisition subsystem is introduced for providing intelligence and considering the most suitable choice models. We develop the feedforward multilayer perceptron artificial neural network for forecasting revenue improvement percent obtained by using more realistic choice models. The obtained results demonstrate new system will decrease the complexity of the system simultaneously with preserving the firm’s revenue. According to the computational results, by increasing the resource restriction, the process of incorporating more realistic choice model will be more important.

Classification of Brainstem Auditory Evoked Potentials Using Artificial Neural Network Based on Time and Frequency Domain Features

This article presents a novel technique of automated classification of brainstem auditory evoked potentials (BAEPs). Brainstem auditory evoked potentials are the evoked potentials used to measure the brain activity that occur in response to the auditory stimulus to the ear and to record the response by placing electrodes on the surface of the scalp. Brainstem auditory evoked potentials reflect neuronal activity in the auditory nerve pathway from the cochlea to the auditory cortex and are used in the diagnosis of neurological diseases leading to hearing loss. In the proposed automated system, the features of BAEPs are extracted in time domain based on latency, and in frequency domain, using fast Fourier transform and discrete wavelet transform. The features extracted from BAEPs are used as variables to build a feed-forward multilayer perceptron, an artificial neural network model for the classification of BAEPs. The stratified 10-fold cross-validation technique is used in training and testing the model to estimate the performance of the system. The combined time and frequency domain features along with physiological parameters as input to artificial neural network resulted in a very good average accuracy of 90.74% in classifying the normal and abnormal BAEP signals.

Neural network-random generation approach to analyze X-ray fluorescence spectra

Application of a feed-forward multilayer perceptron neural network to analyze X-ray fluorescence spectra of a compound material is investigated. For this purpose, a general approach is proposed to improve the learning process of the neural network. In this approach, randomly generated composite spectra are constructed from the set of reference spectra corresponding to the pure elements which compose the material. Instead of taking the spectrum as a whole, the input parameters of the neural network arereduced to the neighboring bins of each pure element peak. Bytesting themethodon the spectra of materials made of Ca, Fe, Ni and Cu elements, we obtained an efficiency of about 95% for predicting the relative concentrations of the four elements in the material. Multivariate Data Analysis refers to any statistical technique used to analyze data that arise from more than one variable.Multivariate classification methods based on machine learning techniques have become a fundamental ingredient to most analyses. The multivariate classifiers themselves have signifi- cantly evolved in recent years. Among the multivariate classification methods, Artificial Neural Networks (ANN) are widely used in various fields. Applications range from image analysis to financial movements predictions and analysis. In (1,8) a review of applications of ANN and its potentialities in atomic and nuclear physics is shown. In high energy physic, ANNis mainly used for classification tasks such assignal over background discrimi- nation. With the search for ever smaller signals in ever larger data sets, it has become essential to extract a maximum of the available information from the data (2). ANN are also applied in other cases where strong non-lineal effects are present like in the spectral analyses generated in analytical techniques like PIXE (3) and XRF (X-ray fluorescence) (4). Several types of artificial neural network are commonly used in the practice. Among these types, we can mention the multi layer perceptrons (MLP), the radial basis function networks (RBF) and theself- organizing maps (SOM). They differ by their processing elements and bythe specific manner in which these elements are connected. The potentialities of ANN seem to be more convenient in the repetitive analysis of many spectra presenting similar patterns with specificdifferences. A typical case is presented in the elementalanalysis of samples of unknown composition using energy dispersiveXRF spectroscopy.Their respective spectra

Simulation of CO2 capture using sodium hydroxide solid sorbent in a fluidized bed reactor by a multi-layer perceptron neural network

Various investigations have been conducted in order to decrease worldwide carbon dioxide in recent decades. Presently, CO2 capture applying solid sorbent has attracted attentions as a manner in which the energy consumption is relatively low. In this study, a feed forward multi-layer perceptron neural network has been developed to predict the ratio of output to input of carbon dioxide concentration (Cout/Cin) in a fluidized bed reactor applied for CO2 capture using sodium hydroxide solid sorbent over operational conditions: temperature (25–40 °C), CO2 volume percentage (1–2%), air flow rate (14–16 m3/hr) and time (0–420 s). The ANN was trained by the Levenberge-Marquardt algorithm, enhanced through the combination with Bayesian regularization technique. Regression analysis results (R2 = 0.9838) and comparison of the ANN predicted Cout/Cin values with corresponding experimental data (%AARD = 1.9217) have shown high prediction ability and robustness of the developed neural network.