Back-propagation Multilayer Perceptron Research Articles

Implementation of Neural Network in Early Detection of Financial Crisis in Singapore

The financial crisis that occurred in 1997 and 2008 had a negative impact on several countries, including Singapore. A financial crisis can occur suddenly so it can endanger a country’s economy if it is not prepared for it. Therefore, early detection of financial crises is needed as a form of crisis warning so that the government can anticipate and prepare appropriate policies. The independent variables used are monthly data of 11 key macroeconomic and financial indicators of Singapore’s economy from January 1990 to June 2021. The Perfect signal is used as the dependent variable in the crisis early detection system. This study aims to build a model of a financial crisis detection system in Singapore using Multilayer Perceptron Backpropagation (MLPBP) as a neural network algorithm by comparing the optimization of Stochastic Gradient Descent (SGD) and Nesterov-accelerated Adaptive Moment Estimation (Nadam). The optimal hyperparameter value in the model was searched using the grid search method based on the accuracy and obtained the best model with 11-11-1 network architecture, best optimization is Nadam, learning rate = 0.1;

Song’s & Non-Song’s Pattern Classification Using Neural Network

The problem of pattern classification between musical or non-musical song audio snippets and non-songs is the main subject of this paper. The machine learning approach accomplishes the goal of automatically interpreting recordings to see if they are songs or not. There are two types of songs: musical and non-musical. This is due to the possibility that it could be difficult to discern between non-musical songs and non-song patterns. Thus, it is necessary to have such systems that differentiate between non-song patterns and songs, whether or not they are melodic. The extraction and selection of characteristics have made advantage of the preprocessing phases. The features that were obtained were pitch, intensity, length (duration), tempo, and sample rate. The Back-propagation Multi-layer Perceptron Neural Network model is used for both dataset testing and model training. This technique will distinguish between a musical or non-musical song and a non-musical piece (such conversation and speech). Training data for the dataset included recordings of speeches and dialogues in Hindi and Urdu for non-song audio files, as well as audio of various male and female singers from Pakistan and India for song audio files in Hindi and Urdu. When the classifier was evaluated using a range of audio samples, 90% of its classifications for songs and non-songs were accurate.

Deriving groundwater major ions from electrical conductivity using artificial neural networks supported by analytical hydrochemical solutions

Tracing groundwater chemistry requires broad and continuous sampling campaigns to keep track of groundwater deterioration. The need is pressing in coastal aquifers due to high water demand to meet rapid population growth. This paper uses artificial intelligence (AI) to develop an algorithm capable of predicting seven major chemical ions in water (Ca2+, Mg2+, Na+, K+, HCO3−, Cl− and SO42−) from electrical conductivity (EC), a single input that can be easily acquired manually or via automated monitoring programs. The algorithm mainly relied on artificial neural network (ANN) to tackle complex and nonlinear relationships. It included a set of procedures among others, Levenberg-Marquardt back propagation multilayer perceptron (MLP), conjugate gradient back propagation with Fletcher-Reeves updates MLP, and support vector regression. The low accuracy of ANN models in predicting Ca2+ and HCO3− concentrations justified their calculation via advanced hydrochemical analytical solutions that were embedded in one AI scheme. The full scheme was trained and tested over an EC20 range of 400–3000 μS/cm, and then validated for new measurements elsewhere. It showed <15% discrepancy for the four ions that constitute the majority of groundwater chemical composition (Cl−, Na+, HCO3− and Ca2+). Higher uncertainties were associated with concentrations derived for EC values outside the training limits. The developed algorithm can save time and money in return of labor and laboratory work. It is an open source code publicly available for further improvements by training it on additional data covering a wider EC range.

A Nonlinear Free Vibration Analysis of Functionally Graded Beams Using a Mixed Finite Element Method and a Comparative Artificial Neural Network

Based on the Hamilton principle combined with the Timoshenko beam theory, the authors developed a mixed finite element (FE) method for the nonlinear free vibration analysis of functionally graded (FG) beams under combinations of simply supported, free, and clamped edge conditions. The material properties of the FG beam gradually and smoothly varied through the thickness direction according to the power-law distributions of the volume fractions in the constituents, and the effective material properties of the FG beam were estimated using the rule of mixtures. The von Kármán geometrical nonlinearity was considered. The FE solutions of the amplitude-frequency relations of the FG beam were obtained using an iterative process. Implementing the mixed FE method showed that its solutions converged rapidly and that the convergent solutions closely agreed with the accurate solutions reported in the literature. A multilayer perceptron (MP) back propagation neural network (BPNN) was also developed to predict the nonlinear free vibration behavior of the FG beam. After appropriate training, the prediction of the MP BPNN’s amplitude-frequency relations was entirely accurate compared to those obtained using the mixed FE method, and its central processing unit time was less time-consuming than that of the mixed FE method.

Predicting Medicine-Stocks by Using Multilayer Perceptron Backpropagation

Artificial neural network has a lot of ability in controlling the error rate to formulate some of its functions as a supervised method. Medicine is one of the major needs for each patient, so that every hospital should know how much inventory of drugs is used and needed by patients every day. This study uses artificial neural network with multilayer perceptron backpropagation as a solution for predicting drug stocks. Prediction of drug stocks using medicine prior period stock data for three years is used to get the predicted results with a small error rate. Backpropagation algorithm using the error output is used to change the weights in the backward direction. To get this error, forward stage should be done first. The results of experiments using backpropagation with the configuration of 0.04 mementum which has training rate of 0.001 gets the value of MSE of 0.00001.

Determination of piers shear capacity using numerical analysis and machine learning for generalization to masonry large scale walls

In this study, comprehensive investigations were conducted on the structural behavior of masonry shear walls considering both aspect ratios and the axial forces. A reliable and simplified macro approach called Multi Pier (MP) method was employed to analyze single masonry piers with different aspect ratios (h/l). The shear and flexural strength along with the failure mechanism of each pier were obtained versus various levels of axial force during the sensitivity analysis. The results of the analysis were used as an input of a machine learning approach called Back Propagation Multi-Layer Perceptron (BPMLP) to generalize the data. The outcomes of machine learning was utilized to develop simple, but accurate equations as well as a user-friendly application to obtain the shear strength of a masonry pier according to the mechanical and geometrical properties. The BPMLP network is also able to produce and estimate all potential failure modes of a masonry wall. Finally, the efficiency of the proposed equations and developed application was evaluated using a novel macro-element approach. It is concluded that the proposed equations can be used in any macro-elements approach to enhancing efficiency and saving time. The proposed Equivalent Wall (EW) model was able to precisely estimate the response of full-scale façade masonry walls subjected to the in-plane loads. According to this model, each pier and spandrel was substituted by two elastic continuous (2D) macro-elements which connected via two bi-component discrete elements. A linear behavior based on masonry modulus of elasticity and bi-linear constitution according to the shear strength was assigned to the axial and shears components of the discrete links, respectively. The proposed EW model was verified using a five-story-seven-bay masonry wall. Satisfactory results were observed in terms of global load-deflection behavior comparing with popular approaches from the literature.

Analysing of nano-silica usage with fly ash for grouts with artificial neural network models

When grout is used to penetrate voids and cracks in soils and rock layers, easy and effective pumping of the grouts is vital, especially for grouting works during geotechnical improvements. For this reason, improving the rheological parameters of cement-based grouts and increasing their fluidity are important for effective grouting injection. In this study, an experimental investigation and analysis using artificial neural network (ANN) models were used to discover how nano silica (n-SiO2) together with fly ash affects the rheological behaviour of cement-based grouts. The effects of nano silica (n-SiO2) additions at different contents by mass (0.0%, 0.3%, 0.6%, 0.9%, 1.2% and 1.5%) on the plastic viscosity and yield stress values of cement-based grouts incorporating fly ash as a mineral additive at different amounts (0% – as a control, 5%, 10%, 15%, 20%, 25% and 30%) were investigated. Using the experimental data obtained, a feed-forward (FF) back-propagation (BP) multi-layer perceptron (MLP) artificial neural network (ANN) was developed to predict the plastic viscosity and yield stress of cement-based grouts with nano silica nanoparticle additives. The ANN model developed can predict the plastic viscosity and yield stress values of cement-based grouts containing nano silica nanoparticle-doped fly ash with high accuracy.

Assessment of Multi-Layer Perceptron Neural Network for Pulmonary Function Test’s Diagnosis Using ATS and ERS Respiratory Standard Parameters

The aim of the research work is to investigate the operability of the entire 23 pulmonary function parameters, which are stipulated by the American Thoracic Society (ATS) and the European Respiratory Society (ERS), to design a medical decision support system capable of classifying the pulmonary function tests into normal, obstructive, restrictive, or mixed cases. The 23 respiratory parameters specified by the ATS and the ERS guidelines, obtained from the Pulmonary Function Test (PFT) device, were employed as input features to a Multi-Layer Perceptron (MLP) neural network. Thirteen possible MLP Back Propagation (BP) algorithms were assessed. Three different categories of respiratory diseases were evaluated, namely obstructive, restrictive, and mixed conditions. The framework was applied on 201 PFT examinations: 103 normal and 98 abnormal cases. The PFT decision support system’s outcomes were compared with both the clinical truth (physician decision) and the PFT built-in diagnostic software. It yielded 92–99% and 87–92% accuracies on the training and the test sets, respectively. An 88–94% area under the receiver operating characteristic curve (ROC) was recorded on the test set. The system exceeded the performance of the PFT machine by 9%. All 23 ATS\ERS standard PFT parameters can be used as inputs to design a PFT decision support system, yielding a favorable performance compared with the literature and the PFT machine’s diagnosis program.

EEG Signal Classification of Motor Imagery Right and Left Hand using Common Spatial Pattern and Multilayer Perceptron Back Propagation

The number of people with disabilities is increasing, so it requires bionic devices to replace human motor functions. Brain-Computer Interface (BCI) can be a tool for the bionic device to communicate with the brain. Signal brain or Electroencephalogram (EEG) signal need to classify to drive the corresponding bionic device. This research goal is to classify the imagination of the right and left-hand movements based on the EEG signal. The system design in this research consists of EEG channel selection using Finite Impulse Response (FIR) filter, feature extraction using Common Spatial Pattern (CSP), and classification using Multilayer Perceptron Back Propagation (MLP-BP). The data used a secondary dataset from BCI Competition IV (2b) with 9 research subjects. The research scenario is carried out by trying to use several variations in the number of hidden layer nodes on each EEG channel. Based on the test, the best accuracy for MLP-BP is 68.7% using 24 nodes in the alpha channel.

Modeling and predicting of water production by capacitive deionization method using artificial neural networks

Water desalination is a method to deal with water shortage that today considered as a way to meet the growing human demand. Obtaining freshwater at low cost guarantees the stability of the desalination method. Capacitive Deionization (CDI) method is a new technology for deionization of water with saline properties, the least energy, and environmental pollution. In this paper, an Artificial Neural Network (ANN) model was developed to estimate the amount of water produced by the CDI method based on experimental data. The backpropagation Multi-Layer Perceptron (MLP) model with Adam (Adaptive Moment Estimation) learning algorithm with two hidden layers was used. To evaluate performance, RMSE, MAE, MSE and R2 mathematical indices were used. The input layer had three variables that include the amount of initial solution concentration, the flow rate, and the amount of voltage applied to the cell. The output layer had a neuron as the amount of percentage of water desalination. For estimating the amount of salt removal percentage, the ANN modeling method was acceptable. The amount of correlation coefficient between experimental laboratory data and data estimated by ANN for training data was equal to 0.972. The overall correlation coefficient was 0.90 that for estimating the amount of salt removal percentage by CDI method is acceptable. The RMSE values for testing and training data were 0.008 and 0.003, respectively. In the statistical study of the effect of variables, it was found that by CDI method the initial concentration of salt has an inverse relationship with the amount of salt removed from water. Also, the amount of applied voltage has a direct relationship and the amount of feed rate has an inverse relationship with the amount of salt removed from water.

Numerical determination of condensation pressure drop of various refrigerants in smooth and micro-fin tubes via ANN method

Abstract In the current work, the pressure drop of the refrigerant flow in smooth and micro-fin pipes has been modeled with artificial neural networks as one of the powerful machine learning algorithms. Experimental analyses have been evaluated in two groups for the numerical model such as operation parameters/physical properties and dimensionless numbers used in two-phase flows. Feed forward back propagation multi-layer perceptron networks have been developed evaluating the practically obtained dataset having 673 data points covering the flow of R22, R134a, R410a, R502, R507a, R32 and R125 in four different pipes. The outputs acquired from the artificial neural network have been evaluated with the target ones, and the performance factors have been estimated and the prediction accuracy of the network models has been resourced comprehensively. The results revealed that the neural networks could predict the pressure drop of the refrigerant flow in smooth and micro-fin pipes between 10% deviation bands.

Machine learning approach to predict the heat transfer coefficients pertaining to a radiant cooling system coupled with mixed and forced convection

Mixed convection phenomenon over radiant cooled surfaces with displacement ventilation in living environments is becoming a popular issue due to the airborne viruses and energy economy. Artificial neural networks are one of the machine learning methods that are widely evaluated as an engineering tool. In the current study, heat transfer coefficients for a radiant wall cooling system coupled with mixed and forced convection have been predicted by a machine learning approach. This approach should be noted as a first experimental investigation couple with an artificial neural network analysis in the open sources in which mixed convection systems in real sized living environments is examined. Experimentally obtained heat transfer coefficients have been used in the development of the feed forward back propagation multi-layer perceptron network structure. So as to analyze the impact of the input factors on the prediction performance, two neural network structures with dissimilar input parameters such as various temperatures, velocities, and heat transfer rates have been developed. By means of feed forward back propagation multi-layer perceptron neural network algorithms, convection, radiation, and total heat transfer coefficients have been predicted using the experimentally acquired dataset including 35 data points belonging to the mixed and forced convection conditions. Training, validation, and test data groups include 70%, 15%, and 15% of the dataset, in turn. Training algorithm has been computed via Levenberg-Marquardt one with 10 neurons in the hidden layer. The findings obtained from the computational solution have been evaluated as a result of the contrast with the target data with in the ±5% deviation band for all heat transfer coefficients. The performance factors have been computed and the estimation precision of the numerical models has been thoroughly examined.

Methodology for integration of wind resource forecasts based on artificial neural networks

An adaptation of the portfolio theory (PT) is proposed in this article, denoted as PrevPT, “Previsão” (in Portuguese) by PT, to integrate the three artificial neural networks, namely multilayer perceptron (MLP) backpropagation, radial basis function (RBF), and self-organizing map (SOM), based forecasting techniques, aiming to analyze the impact of wind speed forecasting errors and achieve more accurate results. In its first use, the PT goal was to maximize a financial return, at any risk, through the diversification of securities or investments that are not positively correlated. Based on the development of PrevPT, which was used until this work only for solar forecasting, the proposed technique is applied in this paper to integrate and improve the results of individual wind forecasts. Four-year wind speed data (January 2007 to December 2010) from two different locations (Algeciras, Spain and Petrolina, Brazil) were used. Our methodology develops a topology that integrates the forecasts obtained by MLP, RBF, and SOM aiming to obtain smaller forecast errors. By diversifying the forecasted asset, when one of the assets has negative prediction errors, another compensates for them and, thus, the total or partial cancellation of the errors occurs. PrevPT obtains a mean absolute percentage error of 1.13% for Spain and 2.35% for Brazil. PrevPT surpassed the results obtained by the three techniques applied individually in the two locations. The main innovations of the methodology are the significant reduction of errors and optimization of resource planning, and the beneficial features compared to other predictor integration techniques.

Analysis of the effect of arrhenius activation energy and temperature dependent viscosity on non-newtonian maxwell nanofluid bio-convective flow with partial slip by artificial intelligence approach

This study focused on the analysis of partial slip effect of Arrhenius activation energy and temperature dependent viscosity on non-Newtonian Maxwell nanofluid bio-convective flow using artificial intelligence approach. Local Nusselt number, local Sherwood number and local density number values, which are dimensionless flow parameters, have been used to examine the said effect. Three different artificial neural network models have been developed using the numerically obtained data sets. Each of the feed forward back propagation multilayer perceptron network models has been developed with different input parameters. 80% of the data set has been used for training the model and 20% for the testing phase. The estimation performance of the network models developed with the Bayesian regularization training algorithm has been extensively analyzed, and the compatibility between the estimation values and the target data has been examined. The findings have shown that artificial neural network models have been developed to make predictions with high accuracy. In addition, artificial neural networks have also proven to be an ideal engineering tool that can be used to analyze the partial slip effect of non-Newtonian Maxwell nanofluids on bio-convective flow.

Non-technical losses in an electrical distribution system. Energy efficiency as a continuous process

This work analyzes the problem of non-technical losses in a low-voltage electrical power distribution system with an artificial intelligence model. The problem focuses on the use of a backpropagation multilayer perceptron neural network, which will quantitatively determine these losses from the manipulation of the current profile and the voltage drop expressed as percentages of their nominal values. These two magnitudes considered, will allow obtaining from the identification of patterns with large variations a technical assessment or quantification to approach the analysis from a different perspective, differing in the methodological determination of non-technical losses currently considered as a percentage of total losses. The results obtained will allow to form a basis for the correct determination of the non-supplied energy, its influence on distribution costs and energy efficiency for the future insertion of distributed generation.

Determination of inactive water quality variables by MODIS data: A case study in the Kızılırmak Delta-Balik Lake, Turkey

This study aims to apply Moderate Resolution Imaging Spectroradiometer (MODIS) data to monitor water quality parameters of total phosphorus (TP) and total nitrogen (TN) in Balik Lake since TP and TN are important factors for eutrophication as well as it is essential to monitor TP and TN concentrations accurately for the management of water environments. Water chemical variables, including TN and TP, are soluble and optically inactive. For such a purpose, remote sensing (RS) enables an alternative way to monitor the optical activity of water with some limitations. In this study, TN and TP concentrations of Balik Lake are examined by RS data. The artificial neural networking (ANN) model was used to reveal the relation between the reflectance bands of MODIS along with the water quality parameters of TP and TN concentrations in aquatic environments by utilizing ground truth water samples collected concurrently with the MODIS overpass. The dataset used in the study consists of MODIS Aqua bands from 1 to 7 and ground truth observations of TP and TN from the study area of Balik Lake during 2017–2019. The development and testing of a feedforward back-propagation multilayer perceptron artificial neural network were performed by using the Levenberg-Marquardt training algorithm to predict TP and TN. The accuracy of the ANN model was determined to be relatively higher in the testing stage of TP prediction with R = 0.59 for Model-1 (3-10-1) and R = 0.56 for Model-2 (7-10-1) while it is lower in the testing stage of TN prediction with R = 0.36 for Model-1 and Model-2. The findings of this study indicated that the reflectance bands of MODIS have a moderate potential to monitor the solubility variables of the water quality for the management of the lake. However, the ANN model introduced in this study is expected to ensure an improvement in the prediction accuracy of inactive water quality variables based on the RS data.

Biomechanical Study and Prediction of Lower Extremity Joint Movements Using Bayesian Regularization-Based Backpropagation Neural Network

Abstract This work aims to estimate the lower-limb joint angles in the sagittal plane using Microsoft Kinect-based experimental setup and apply an efficient machine learning technique for predicting the same based on kinematic, spatiotemporal, and biological parameters. Ten healthy participants from 19 to 50 years (33 ± 11.24 years) were asked to walk in front of the Kinect camera. Based on the skeleton image, the biomechanical hip, knee, and ankle joint angles of the lower-limb were measured using ni-labview. Thereafter, two Bayesian regularization-based backpropagation multilayer perceptron neural network models were designed to predict the joint angles in the stance and swing phase. The joint angles of two individuals, as a testing dataset, were predicted and compared with the experimental results. The test correlation coefficient for predicted joint angles has shown a promising effect of the proposed neural network models. Finally, a qualitative comparison was presented between the joint angles of healthy people and unhealthy people of similar age groups.

Air Quality Modeling for Sustainable Clean Environment Using ANFIS and Machine Learning Approaches

Air quality monitoring and assessment are essential issues for sustainable environmental protection. The monitoring process is composed of data collection, evaluation, and decision-making. Several important pollutants, such as SO2, CO, PM10, O3, NOx, H2S, location, and many others, have important effects on air quality. Air quality should be recorded and measured based on the total effect of pollutants that are collectively prescribed by a numerical value. In Canada, the Air Quality Health Index (AQHI) is used which is one numerical value based on the total effect of some concentrations. Therefore, evolution is required to consider the complex, ill-defined air pollutants, hence several naive and noble approaches are used to study AQHI. In this study, three approaches such as hybrid data-driven ANN, nonlinear autoregressive with external (exogenous) input (NARX) with a neural network, and adaptive neuro-fuzzy inference (ANFIS) approaches are used for estimating the air quality in an urban area (Jeddah city—industrial zone) for public health concerns. Over three years, 1771 data were collected for pollutants from 1 June 2016 until 30 September 2019. In this study, the Levenberg-Marquardt (LM) approach was employed as an optimization method for ANNs to solve the nonlinear least-squares problems. The NARX employed has a two-layer feed-forward ANN. On the other hand, the back-propagation multi-layer perceptron (BPMLP) algorithm was used with the steepest descent approach to reduce the root mean square error (RMSE). The RMSEs were 4.42, 0.0578, and 5.64 for ANN, NARX, and ANFIS, respectively. Essentially, all RMSEs are very small. The outcomes of approaches were evaluated by fuzzy quality charts and compared statistically with the US-EPA air quality standards. Due to the effectiveness and robustness of artificial intelligent techniques, the public’s early warning will be possible for avoiding the harmful effects of pollution inside the urban areas, which may reduce respiratory and cardiovascular mortalities. Consequently, the stability of air quality models was correlated with the absolute air quality index. The findings showed notable performance of NARX with a neural network, ANN, and ANFIS-based AQHI model for high dimensional data assessment.

Analysis of Twitter Users Sentiment against the Covid-19 Outbreak Using the Backpropagation Method with Adam Optimization

This research tries to take advantage of Twitter by analyzing Indonesian-language tweets that discuss the Covid-19 virus outbreak to find out what Twitter users think about the Covid-19 virus outbreak. This study tries to analyze sentiment to see opinions on Covid-19 tweets that contains Posittive, Negative or Neutral sentiments using Multi-layer Perceptron (MLP) using Backprogragation with Adam optimization. We collected 200 documents (tweets) in Indonesian about Covid-19 that were tweeted since November 2019 and then trained them to get our MLP Backpropagation model. Our model managed to get an accuracy of up to 70% with f1-scores for positive, negative, and neutral classes respectively 0.77, 0.75, and 0.5 from a maximum value of 1. This shows that our model is quite successful in carrying out the sentiment classification process for Indonesian tweets with the Covid-19 theme.

Real-Time Identification of Fuzzy PID-Controlled Maglev System using TLBO-Based Functional Link Artificial Neural Network

In this paper, the teaching–learning-based optimization-based functional link artificial neural network (FLANN) has been proposed for the real-time identification of Maglev system. This proposed approach has been compared with some of the other state-of-the-art approaches, such as multilayer perceptron–backpropagation, FLANN least mean square, FLANN particle swarm optimization and FLANN black widow optimization. Further, the real-time Maglev system and the identified model are controlled by the Fuzzy PID controller in a closed loop system with proper choice of the controller parameters. The efficacy of the identified model is investigated by comparing the response of both the real-time and identified Fuzzy PID-controlled Maglev system. To validate the dominance of the proposed model, three nonparametric statistical tests, i.e., the sign test, Wilcoxon signed-rank test and Friedman test, are also performed.