Multilayer Perceptron Neural Network Architecture Research Articles

Measurement and diagnostic system for detecting and classifying faults in the rotary hay tedder using multilayer perceptron neural networks

The implementation of preventive maintenance measures, which aim to assure the optimal performance of rotary machines and minimize the occurrence of substantial damage, has gained significant importance in the agricultural machinery industry. As part of this work, several measurements and computations were conducted to develop effective methods for detecting and classifying machine damages based on raw measurement data from accelerometers acquired both in the laboratory and real-life conditions. For this purpose a rotary hay tedder was equipped with a self-developed measuring and diagnostic system. The hardware and network communication components of this system were described. Machine learning methods were applied and the multi-layer perceptron (MLP) neural-network based models were created. The diagnostic models were made using different machine learning algorithms with 90 different configurations of neural-networks in total, including 30 different hidden layer arrangements and three activation functions (relu, softmax and sigmoid). Several learning scenarios were considered in order to determine and to distinguish between a healthy state and a machine failure, such as a missing or broken tedder tine. Based on laboratory experiments and meadow tests, 20 best metrics were selected for the input feature vector. When conducting laboratory and on-site tests, the best efficiency was achieved for the multi-layer perceptron neural-network architecture with two hidden layers (size of 20 and 20) and the relu activation function, and the Adam optimizer. Accuracy, as the ratio of correct predictions to total predictions made, for different diagnostic models was estimated. The confusion matrices were given to present the results for 10 rotors. Data efficiency for different diagnostic models and hyperparameters was also presented. For tests on the meadow, the average accuracy was 79% for detecting a missing tine, and 69% for detecting a tine failure (a missing or broken tine). In the case of the laboratory-based experimental results, the average accuracy for detecting a broken tine in the rotor was 92.5%. Therefore, based on the results, the proposed approach can be used for rotary hay tedder fault diagnosis.

Comparison of multilayer perceptron neural network architecture in photovoltaic plants fault classification

The goal of this study is to assess the application of Multilayer Perceptron Artificial Neural Networks in fault classification within photovoltaic panels, focusing on key characteristics such as the number of neurons and layers, activation functions, training techniques, and the resulting accuracy. The study employs a comparative analysis approach, examining various characteristics and hyperparameters applied to Multilayer Perceptron Artificial Neural Networks for fault classification in photovoltaic panels. The research methodology involves reviewing publications from the past decade to gather data on these characteristics and their impact on fault analysis in photovoltaic generation systems. This study contributes to the originality of the field by providing a comprehensive comparison of various parameters and techniques used in Multilayer Perceptron Artificial Neural Networks for fault classification in photovoltaic panels. The findings offer valuable insights for researchers and practitioners in the renewable energy sector, aiding in the development of more efficient and reliable fault diagnosis systems for photovoltaic generation.

Effect of Textural Properties on the Degradation of Bisphenol from Industrial Wastewater Effluent in a Photocatalytic Reactor: A Modeling Approach

Conventional treatment methods such as chlorination and ozonation have been proven not to be effective in eliminating and degrading contaminants such as Bisphenol A (BPA) from wastewater. Hence, the degradation of BPA using a photocatalytic reactor has received a lot of attention recently. In this study, a model-based approach using a multilayer perceptron neural network (MLPNN) coupled with back-propagation, as well as support vector machine regression coupled with cubic kernel function (CSVMR) and Gaussian process regression (EQGPR) coupled with exponential quadratic kernel function, were employed to model the relationship between the textural properties such as pore volume (Vp), pore diameter (Vd), crystallite size, and specific surface area (SBET) of erbium- and iron-modified TiO2 photocatalysts in degrading BPA. Parametric analysis revealed that effective degradation of the Bisphenol up to 90% could be achieved using photocatalysts having textural properties of 150 m2/g, 8 nm, 7 nm, and 0.36 cm3/g for SBET, crystallite size, particle diameter, and pore volume, respectively. Fifteen architectures of the MPLNN models were tested to determine the best in terms of predictability of BPA degradation. The performance of each of the MLPNN models was measured using the coefficient of determination (R2) and root mean squared errors (RMSE). The MLPNN architecture comprised of 4 input layers, 14 hidden neurons, and 3 output layers displayed the best performance with R2 of 0.902 and 0.996 for training and testing. The 4-14-3 MLPNN robustly predicted the BPA degradation with an R2 of 0.921 and RMSE of 4.02, which is an indication that a nonlinear relationship exists between the textural properties of the modified TiO2 and the degradation of the BPA. The CSVRM did not show impressive performance as indicated by the R2 of 0.397. Therefore, appropriately modifying the textural properties of the TiO2 will significantly influence the BPA degradability.

Training of Classification Models via Federated Learning and Homomorphic Encryption.

With the rise of social networks and the introduction of data protection laws, companies are training machine learning models using data generated locally by their users or customers in various types of devices. The data may include sensitive information such as family information, medical records, personal habits, or financial records that, if leaked, can generate problems. For this reason, this paper aims to introduce a protocol for training Multi-Layer Perceptron (MLP) neural networks via combining federated learning and homomorphic encryption, where the data are distributed in multiple clients, and the data privacy is preserved. This proposal was validated by running several simulations using a dataset for a multi-class classification problem, different MLP neural network architectures, and different numbers of participating clients. The results are shown for several metrics in the local and federated settings, and a comparative analysis is carried out. Additionally, the privacy guarantees of the proposal are formally analyzed under a set of defined assumptions, and the added value of the proposed protocol is identified compared with previous works in the same area of knowledge.

Hybrid model to improve the river streamflow forecasting utilizing multi-layer perceptron-based intelligent water drop optimization algorithm

Artificial intelligence (AI) models have been effectively applied to predict/forecast certain variable in several engineering applications, in particular, where this variable is highly stochastic in nature and complex to identify utilizing classical mathematical model, such as river streamflow. However, the existing AI models, such as multi-layer perceptron neural network (MLP-NN), are basically incomprehensible and facing problem when applied for time series prediction or forecasting. One of the main drawbacks of the MLP-NN model is the ability of the used default optimization algorithm [gradient decent algorithm (GDA)] to search for the optimal weight and bias values associated with each neuron within the MLP-NN architecture. In fact, GDA is a first-order iteration algorithm that usually trapped in local minima, especially when the time series is highly stochastic as in the river streamflow historical records. As a result, the overall performance of the MLP-NN model experienced inaccurate prediction or forecasting for the desired output. Moreover, due to the possibility of overfitting with MLP model which may lead to poor performance of prediction of the unseen input pattern, there is need to introduce new augmented algorithm capable of identifying the complexity of streamflow data and improve the prediction accuracy. Therefore, in this study, a replacement for the GDA with advanced optimization algorithm, namely intelligent water drop (IWD), is proposed to enhance the searching procedure for the global optima. The new proposed forecasting model is, namely MLP-IWD. Two different historical rivers streamflow data have been collected from Nong Son and Thanh My stations on the Vu Gia Thu Bon river basin for period between (1978 and 2016) in order to examine the performance of the proposed MLP-IWD model. In addition, in order to evaluate the performance of the proposed MLP-IWD model under different conditions, four different scenarios for the model input–output architecture have been investigated. Results showed that the proposed MLP-IWD model outperformed the classical MLP-NN model and significantly improve the forecasting accuracy for the river streamflow. Finally, the proposed model could be generalized and applied in different rivers worldwide.

Bayesian Regularization-Trained Multi-layer Perceptron Neural Network Predictive Modelling of Phenol Degradation using ZnO/Fe2O3 photocatalyst

The processing of crude oil in the onshore platform often results in the generation of produce water containing harmful organic pollutants such as phenol. If the produce water is not properly treated to get rid of the organic pollutants, human exposure when discharged could be detrimental to health. Photocatalytic degradation of the organic pollutant has been a proven, non-expensive techniques of removing these harmful organic compounds from the produce water. However, the detail experimentation is often tedious and costly. One way to investigate the non-linear relationship between the parameters for effective performance of the photodegradation is by artificial neural network modelling. This study investigates the predictive modelling of photocatalytic phenol degradation from crude oil wastewater using Bayesian regularization-trained multilayer perceptron neural network (MLPNN). The ZnO/Fe2O3 photocatalyst used for the photodegradation was prepared using sol-gel method and employed for the phenol degradation study in a batch reactor under solar irradiation. Twenty-six datasets generated by Box-Behken experimental design was used for the training of the MLPNN with input variables as irradiation time, initial phenol concentration, photocatalyst dosage and the pH of the solution while the output layer consist of phenol degradation. Several MLPNN architecture was tested to obtain an optimized 4 5 1 configuration with the least mean standard error (MSE) of 1.27. The MLPNN with the 4 5 1 architecture resulted in robust prediction of phenol degradation from the wastewater with coefficient of determination (R) of 0.999.

Development of an artificial neural network model to simulate the growth of microalga Chlorella vulgaris incorporating the effect of micronutrients

Artificial neural network (ANN) models can be trained to simulate the dynamic behavior of biological systems. In the present study, an ANN model was developed upon multilayer perceptron neural network architecture with 23-20-1 configuration to predict the cell concentration of microalga Chlorella vulgaris at a given time. Irradiance level, photoperiod, temperature, air flow rate, CO2 percentage of the air stream, initial cell concentration, cultivation time and the nutrient concentrations of the media were considered as the input variables of the model. Resilient backpropagation learning algorithm was used to train the model by means of 484 experimental data belonging to four studies. Bias and accuracy factors of the developed model fall into the range of 0.95–1.11 indicating the model has an excellent prediction ability. Parity plot showed a good agreement between the predicted and experimental values with R2 = 0.98. Relative importance of the inputs was evaluated using Garson’s algorithm. The results of the study indicated that CO2 supply had the highest impact on the growth of C. vulgaris within the selected range of input parameters. Among macronutrients and micronutrients, highest influence was demonstrated by nitrogen and copper respectively.

Rheological wall slip velocity prediction model based on artificial neural network

ABSTRACTWall slip is a phenomenon in which particles migrate from solid boundaries, leaving a thin liquid rich layer adjacent to a wall, which can affect the measurement of the rheological properties. Currently, analyses of wall slip are normally carried out through experimental study using a rheometer. These traditional methods are generally time consuming, as several experiment sets are usually required. The aim of this research is to develop an alternative, more efficient approach, by formulating a mathematical model able to predict the wall slip velocity with an acceptable level of accuracy. Specifically, this study investigates a Multi-Layer Perceptron Neural Network (MLP-NN) as an advanced method to predict wall slip velocity. It develops and tests several MLP-NN architectures that accommodate a range of fixed input variables including shear stress, concentration, temperature and particle sizes, with estimated wall slip velocity as the output variable. Using this method, users can perform wall slip velocity analyses by simply plugging different patterns of the proposed input variables into the recommended architecture. Our tests show an MLP-NN model with one hidden layer consisting of nine hidden neurons to be the best architecture for such purposes, producing a strong overall performance with an R2 value of 0.9994 and maximum error of 28%. This research study is innovative in its use of artificial intelligence to predict wall slip velocity in rheological applications.

Week-ahead Rainfall Forecasting Using Multilayer Perceptron Neural Network

Accurate rainfall forecasting plays a significant role for weather stations as it serves to warn people about incoming natural disasters. This paper presents an implementation of week-ahead rainfall forecast that utilizes Multilayer Perceptron Neural Network (MLPNN) in processing historical rainfall data. Proper data preparation, model implementation and performance evaluation were conducted to two MLPNN models which yields promising results in predicting week-ahead rainfall. The MLPNN architecture was a supervised feed-forward neural network having 11 input neurons consisting of different weather variables along with various hidden neurons and 7 output neurons representing the week-ahead forecast. The MLPNN models which were SCG-Tangent and SCG-Sigmoid, produced a MAE of 0.01297 and 0.1388 and RMSE of 0.01512 and 0.01557, respectively. This viable implementation of MLPNN in rainfall forecasting hopes to provide organizations and individuals with lead-time for the strategic and tactical planning of activities and courses of action related to rainfall.

Artificial Neural Network Approach to Mobile Location Estimation in GSM Network

AbstractThe increase in utilisation of mobile location-based services for commercial, safety and security purposes among others are the key drivers for improving location estimation accuracy to better serve those purposes. This paper proposes the application of Levenberg Marquardt training algorithm on new robust multilayered perceptron neural network architecture for mobile positioning fitting for the urban area in the considered GSM network using received signal strength (RSS). The key performance metrics such as accuracy, cost, reliability and coverage are the major points considered in this paper. The technique was evaluated using real data from field measurement and the results obtained proved the proposed model provides a practical positioning that meet Federal Communication Commission (FCC) accuracy requirement.

Artificial neural network modelling of cutting force components in milling

The following paper will give an account of experimental tests and simulation of the cutting force components Fx, Fy and Fz in down milling of AZ91D magnesium alloy. The milling operation employed two milling cutters with a different helix angle, λs = 20° and λs = 50°, and was conducted with changeable milling machining parameters: cutting speed, feed per tooth, axial depth of cut. The simulation part of the study was conducted in Statistica software environment with the application of Multi-Layered Perceptron neural network architecture, and using a “black box” approach, which guarantees a good fit of input and output data obtained from the experimental tests.

Integrated versus isolated scenario for prediction dissolved oxygen at progression of water quality monitoring stations

Abstract. This study examined the potential of Multi-layer Perceptron Neural Network (MLP-NN) in predicting dissolved oxygen (DO) at Johor River Basin. The river water quality parameters were monitored regularly each month at four different stations by the Department of Environment (DOE) over a period of ten years, i.e. from 1998 to 2007. The following five water quality parameters were selected for the proposed MLP-NN modelling, namely; temperature (Temp), water pH, electrical conductivity (COND), nitrate (NO3) and ammonical nitrogen (NH3-NL). In this study, two scenarios were introduced; the first scenario (Scenario 1) was to establish the prediction model for DO at each station based on five input parameters, while the second scenario (Scenario 2) was to establish the prediction model for DO based on the five input parameters and DO predicted at previous station (upstream). The model needs to verify when output results and the observed values are close enough to satisfy the verification criteria. Therefore, in order to investigate the efficiency of the proposed model, the verification of MLP-NN based on collection of field data within duration 2009–2010 is presented. To evaluate the effect of input parameters on the model, the sensitivity analysis was adopted. It was found that the most effective inputs were oxygen-containing (NO3) and oxygen demand (NH3-NL). On the other hand, Temp and pH were found to be the least effective parameters, whereas COND contributed the lowest to the proposed model. In addition, 17 neurons were selected as the best number of neurons in the hidden layer for the MLP-NN architecture. To evaluate the performance of the proposed model, three statistical indexes were used, namely; Coefficient of Efficiency (CE), Mean Square Error (MSE) and Coefficient of Correlation (CC). A relatively low correlation between the observed and predicted values in the testing data set was obtained in Scenario 1. In contrast, high coefficients of correlation were obtained between the observed and predicted values for the test sets of 0.98, 0.96 and 0.97 for all stations after adopting Scenario 2. It appeared that the results for Scenario 2 were more adequate than Scenario 1, with a significant improvement for all stations ranging from 4 % to 8 %.

Neural Computation of Wide Aperture Dimension of Optimum Gain Pyramidal Horn

A new method based on the multilayered perceptron neural network architecture for computing the wide aperture dimension of the pyramidal horn is presented. The computed wide aperture dimension is used in successfully designing optimum gain pyramidal horn. The other design parameters of the horn are determined from the simple and explicit analytical formulas. These formulas do not need the application of the iterative methods, and are not restricted to the high gain horn designs. The gain of a designed pyramidal horn is determined with no path length error approximation. Better accuracy with respect to the previous design methods is obtained for various pyramidal horn design examples.

Artificial neural networks for calculating the association probabilities in multi-target tracking

A simple method based on the multilayered perceptron neural network architecture for calculating the association probabilities used in target tracking is presented. The multilayered perceptron is trained with the Levenberg–Marquardt algorithm. The tracks estimated by using the proposed method for multiple targets in cluttered and non-cluttered environments are in good agreement with the original tracks. Better accuracy is obtained than when using the joint probabilistic data association filter or the cheap joint probabilistic data association filter methods.

New approach for modelling ultrasound blood backscatter signal

The ultrasound (US) scattered signal from blood has been treated as a random signal by many investigators. However, the degree of randomness of a medium is a relative term that can change considerably with the resolution of the sensor. In this study, the backscattered signal from blood has been looked at as a chaotic signal. By this treatment, according to Taken’s theorem, a single variable ( e.g., amplitude of the blood-backscattered signal) can be used to reconstruct the nonlinear dynamics of the blood-scattered signal. Multilayer perceptron neural network architecture, with error back-propagation, has been formulated and used as a basis for building and testing the chaotic model of the backscattered signal. This chaotic model is used successfully as a short-term predictor of the backscattered signal from blood-mimicking fluid (BMF) flowing in a vascular flow phantom under pulsatile flow. This modelling approach can be useful, for example, in detecting blood-borne emboli. (E-mail:mbrawany@sghms.ac.uk)

SPEAKER NORMALIZATION IMPROVEMENT BY NEURAL NETWORK PARAMETER OPTIMIZATION

In this work we investigate the use of a speaker adaptation technique, for speech recognition, based on neural network spectral mapping. Different multilayer perceptron neural network architectures are analyzed in order to optimize the spectral difference reduction in acoustic data of two speakers. Experiments are carried out in a telecontrol environment, used to provide voice commands to a mobile robot, based on DTW pattern matching.

MLP iterative construction algorithm

This paper presents a novel multi-layer perceptron neural network architecture selection and weight training algorithm for classification problems. The MLP iterative construction algorithm (MICA) autonomously constructs an MLP neural network as it trains. Experimental results show the algorithm achieves 100% accuracy on the training data, the same or better generalization accuracies as Backprop on the test data, while using less FLOPS. Moreover, relaxation of the hidden layer nodes improves test set recognition accuracies to be greater than that of Backprop. Furthermore, seeding the Backprop algorithm with the hidden layer weights from MICA is demonstrated. The MICA seeding improves the effectiveness of Backprop and enables Backprop to solve a new class of problems, i.e., problems with areas of low mean-squared error.

Computer-aided breast cancer detection and diagnosis of masses using difference of Gaussians and derivative-based feature saliency.

A new model-based vision (MBV) algorithm is developed to find regions of interest (ROI's) corresponding to masses in digitized mammograms and to classify the masses as malignant/benign. The MBV algorithm is comprised of five modules to structurally identify suspicious ROI's, eliminate false positives, and classify the remaining as malignant or benign. The focus of attention module uses a difference of Gaussians (DoG) filter to highlight suspicious regions in the mammogram. The index module uses tests to reduce the number of nonmalignant regions from 8.39 to 2.36 per full breast image. Size, shape, contrast, and Laws texture features are used to develop the prediction module's mass models. Derivative-based feature saliency techniques are used to determine the best features for classification. Nine features are chosen to define the malignant/benign models. The feature extraction module obtains these features from all suspicious ROI's. The matching module classifies the regions using a multilayer perceptron neural network architecture to obtain an overall classification accuracy of 100% for the segmented malignant masses with a false-positive rate of 1.8 per full breast image. This system has a sensitivity of 92% for locating malignant ROI's. The database contains 272 images (12 b, 100 microm) with 36 malignant and 53 benign mass images. The results demonstrate that the MBV approach provides a structured order of integrating complex stages into a system for radiologists.

Investigation into effects of device variation on performance of optoelectronic neural networks

Free space optical links, using LEDs and photodiodes, have been studied as a means of communication between neural layers in an analogue multilayered perceptron neural network architecture, trained using the back-propagation learning algorithm. Simulation results on network learning and generalisation show that such networks are tolerant to as much as 75% variation in either the LED or photodiode responses.

Frequency division multiplexing in analogue neural network

Frequency division multiplexing has been studied as a means of communication between neural layers in an analogue multilayered perceptron neural network architecture, trained using the back-propagation learning algorithm. Simulation results on network learning and generalisation show that the neural network is tolerant to as much as 50% overlap of frequency responses of filters used in demultiplexing. Thus, the number of communication channels available is considerably increased.