Conjugate Gradient Training Algorithm Research Articles

Forecasting the Financial Soundness of Indonesia’s National Board of Zakat (BAZNAS) Using Artificial Neural Network Models

The potential value of zakat in Indonesia in 2021 was estimated at RP 13.529 trillion; however, the actual amount of zakat collected was only RP 571 billion. Thus, this study aims to develop a precise financial soundness forecasting model for the National Board of Zakat (BAZNAS) in Indonesia using multiple linear regression (MLR) and artificial neural networks (ANN) with three algorithms to investigate, compare, and interpret the obtained results for better forecasting. The information was extracted from the financial statements of six BAZNAS for the period spanning year 2017 to 2021. The study determined unique and best models for each current ratio, cash ratio, and quick ratio. The results highlighted two models deemed best for forecasting Indonesia's BAZNAS financial soundness: MLR and ANN with Scaled Conjugate Gradient (SCG) training algorithm and MLR and ANN with Bayesian Regularization (BR). The research implications could help decision-makers strategise for the financial health of other Zakat organisations accordingly.

Application of Non-Destructive Test Results to Estimate Rock Mechanical Characteristics—A Case Study

Accurately determining rock elastic modulus (EM) and uniaxial compressive strength (UCS) using laboratory methods requires considerable time and cost. Hence, the development of models for estimating the mechanical properties of rock is a very attractive alternative. The current research was conducted to predict the UCS and EM of sandstone rocks using quartz%, feldspar%, fragments%, compressional wave velocity (PW), the Schmidt hardness number (SN), porosity, density, and water absorption via simple regression, multivariate regression (MVR), K-nearest neighbor (KNN), support vector regression (SVR) with a radial basis function, the adaptive neuro-fuzzy inference system (ANFIS) using the Gaussian membership (GM) function, and the back-propagation neural network (BPNN) based on various training algorithms. The samples were categorized as litharenite and feldspathic litharenite. By increasing the feldspar% and quartz% and decreasing the fragments%, the static properties increased. The results of the statistical analysis showed that the SN and porosity have the greatest effect on the UCS and EM, respectively. Among the Levenberg–Marquardt (LM), Bayesian regularization, and Scaled Conjugate Gradient training algorithms using the BPNN method, the LM achieved the best results in forecasting the UCS and EM. The ideal obtained BPNN, using a trial-and-error process, contains four neurons in a hidden layer with eight inputs. All five models attained acceptable accuracy (correlation coefficient greater than 70%) for estimating the static properties. By comparing the methods, the ANFIS showed higher precision than the other methods. The UCS and EM of the samples can be determined with very high accuracy (R2 > 99%).

Predicting dry matter intake in Canadian Holstein dairy cattle using milk mid-infrared reflectance spectroscopy and other commonly available predictors via artificial neural networks.

Dry matter intake (DMI) is a fundamental component of the animal's feed efficiency, but measuring DMI of individual cows is expensive. Mid-infrared reflectance spectroscopy (MIRS) on milk samples could be an inexpensive alternative to predict DMI. The objectives of this study were (1) to assess if milk MIRS data could improve DMI predictions of Canadian Holstein cows using artificial neural networks (ANN); (2) to investigate the ability of different ANN architectures to predict unobserved DMI; and (3) to validate the robustness of developed prediction models. A total of 7,398 milk samples from 509 dairy cows distributed over Canada, Denmark, and the United States were analyzed. Data from Denmark and the United States were used to increase the training data size and variability to improve the generalization of the prediction models over the lactation. For each milk spectra record, the corresponding weekly average DMI (kg/d), test-day milk yield (MY, kg/d), fat yield (FY, g/d), and protein yield (PY, g/d), metabolic body weight (MBW), age at calving, year of calving, season of calving, days in milk, lactation number, country, and herd were available. The weekly average DMI was predicted with various ANN architectures using 7 predictor sets, which were created by different combinations MY, FY, PY, MBW, and MIRS data. All predictor sets also included age of calving and days in milk. In addition, the classification effects of season of calving, country, and lactation number were included in all models. The explored ANN architectures consisted of 3 training algorithms (Bayesian regularization, Levenberg-Marquardt, and scaled conjugate gradient), 2 types of activation functions (hyperbolic tangent and linear), and from 1 to 10 neurons in hidden layers). In addition, partial least squares regression was also applied to predict the DMI. Models were compared using cross-validation based on leaving out 10% of records (validation A) and leaving out 10% of cows (validation B). Superior fitting statistics of models comprising MIRS information compared with the models fitting milk, fat and protein yields suggest that other unknown milk components may help explain variation in weekly average DMI. For instance, using MY, FY, PY, and MBW as predictor variables produced a predictive accuracy (r) ranging from 0.510 to 0.652 across ANN models and validation sets. Using MIRS together with MY, FY, PY, and MBW as predictors resulted in improved fitting (r = 0.679-0.777). Including MIRS data improved the weekly average DMI prediction of Canadian Holstein cows, but it seems that MIRS predicts DMI mostly through its association with milk production traits and its utility to estimate a measure of feed efficiency that accounts for the level of production, such as residual feed intake, might be limited and needs further investigation. The better predictive ability of nonlinear ANN compared with linear ANN and partial least squares regression indicated possible nonlinear relationships between weekly average DMI and the predictor variables. In general, ANN using Bayesian regularization and scaled conjugate gradient training algorithms yielded slightly better weekly average DMI predictions compared with ANN using the Levenberg-Marquardt training algorithm.

Badminton stroke identification using wireless inertial sensor and neural network

Objective evaluation is essential in sports to monitor athlete performance, provide relevant and timely feedback, and minimize the risk of injury. Activity recognition is the first step in sport skill and technique performance analysis. This study investigated the use of wearable inertial sensors and a neural network (NN) to identify badminton strokes. The study also explored the effect of different NN configurations and a different number of sensors on the classification. Sensors were placed at the dominant wrist, left ankle, and right ankle. Six different strokes, ranging from soft hitting net shots to smashes, were performed with a total of 3300 repetitions from six well-trained badminton players. An automated window segmentation method was developed to identify the stroke instances. A scaled conjugate gradient training algorithm with two hidden layers and 55 neurons in each layer was found to be the best approach to classify badminton strokes with an accuracy of 97.69%. Even just wearing the inertial sensor on the wrist was sufficient, providing an accuracy of 95.09%. These results demonstrate the viability of using inertial sensors and NN to recognize badminton strokes, which can be applied in training and competitive environments.

Comparison of different artificial neural network (ANN) training algorithms to predict the atmospheric temperature in Tabuk, Saudi Arabia

Use of Artificial neural network (ANN) models to predict weather parameters has become important over the years. ANN models give more accurate results in weather and climate forecasting among many other methods. However, different models require different data and these data have to be handled accordingly, but carefully. In addition, most of these data are from non-linear processes and therefore, the prediction models are usually complex. Nevertheless, neural networks perform well for non-linear data and produce well acceptable results. Therefore, this study was carried out to compare different ANN models to predict the minimum atmospheric temperature and maximum atmospheric temperature in Tabuk, Saudi Arabia. ANN models were trained using eight different training algorithms. BFGS Quasi Newton (BFG), Conjugate gradient with Powell-Beale restarts (CGB), Levenberg-Marquadt (LM), Scaled Conjugate Gradient (SCG), Fletcher-Reeves update Conjugate Gradient algorithm (CGF), One Step Secant (OSS), Polak-Ribiere update Conjugate Gradient (CGP) and Resilient Back-Propagation (RP) training algorithms were fed to the climatic data in Tabuk, Saudi Arabia. The performance of the different training algorithms to train ANN models were evaluated using Mean Squared Error (MSE) and correlation coefficient (R). The evaluation shows that training algorithms BFG, LM and SCG have outperformed others while OSS training algorithm has the lowest performance in comparison to other algorithms used.

Carbon dioxide reforming of methane over Ni-based catalysts: Modeling the effect of process parameters on greenhouse gasses conversion using supervised machine learning algorithms

This study aims to model the effect of process parameters on the conversion of carbon dioxide (CO2) and methane (CH4) during reforming reaction over Nickel (Ni) catalysts. Various supervised machine learning algorithms were employed for the model development. To determine the best model, different configurations of the multilayer perceptron (MLP) and nonlinear auto-regressive exogenous (NARX) neural network models and their performances were evaluated. The performance of the various models was tested through their ability to predict the conversion of the CO2 and CH4. The best MLP network configurations of 5–15–2, 5–4–2, and 5–7–2 were obtained for the Levenberg-Marquardt-, the Bayesian Regularization-, and the Scaled conjugate gradient-trained MLP, respectively. While optimized NARX neural network configurations of 5–18–2, 5–13–2, and 5–8–2 were obtained for the Levenberg-Marquardt, Bayesian Regularization, and the Scaled conjugate gradient training algorithms, respectively. The Bayesian Regularization trained NARX with a coefficient of determination (R2) of 0.998 and MSE of 3.24×10–9 displayed the best performance with an accurate prediction of the thermo-catalytic conversion of CH4 and CO2. The sensitivity analysis revealed that the predicted CH4 and CO2 conversion were influenced in the order of reaction temperature > reduction temperature > calcination temperature > time on stream > Ni loading.

Performance Analysis of Traffic Congestion Using Designated Neural Network Training Algorithms

A lot of neural network training algorithms on prediction exist and these algorithms are being used by researchers to solve evaluation, forecasting, clustering, function approximation etc. problems in traffic volume congestion. This study is aimed at analysing the performance of traffic congestion using some designated neural network training algorithms on traffic flow in some selected corridors within Akure, Ondo state, Nigeria. The selected corridors were Oba Adesida road, Oyemekun road and Oke Ijebu road all in Akure. The traffic flow data were collected manually with the help of field observers who monitored and record traffic movement along the corridors. To accomplish this, three common training algorithms were selected to train the traffic flow data. The data were trained using Bayesian Regularization (BR), Scaled Conjugate Gradient (SCG) and Levenberg-Marquardt (LM) training algorithms. The outputs/performances of these training functions were evaluated by using the Mean Square Error (MSE) and Coefficient of Regression (R) to find the best training algorithms. The results show that, the Bayesian regularization algorithm, performs better with MSE of 2.37e-13 and R of 0.9999 than SCG and LM algorithms.

Artificial neural network prediction of CO emission and ash yield from co-combustion of empty fruit bunch, palm kernel shell and kaolin.

The quantity of ash yield and carbon monoxide (CO) emitted during co-combustion of empty fruit bunch (EFB), palm kernel shells (PKS) and kaolin in a grate furnace depend on the fuels mixing ratio, the combustion temperature and duration. These factors can be tuned to minimize ash deposition and CO emission which is partly responsible for the greenhouse effect. In this study, seventy-three (73) data points were obtained from combustion of EFB, PKS and kaolin mixtures based on D-optimal design. Artificial neural network (ANN) model, optimized with Taguchi technique, was developed to predict ash yield (AY) and CO emission from the combustion of the fuel mixture. The data were divided into training, validation and testing in a 2:1:1 relative proportion. The optimized ANN architecture for AY and CO emission were 5-11-3-1 and 5-6-3-1, respectively, with scale conjugate gradient training algorithm and a learning rate of 0.1. Results of the ANN model agreed significantly with the experimental results with coefficients of determination (R2) of 0.96 and 0.93 for ash yield and CO emission, respectively. The mathematical models for the ash and CO emission using the D-optimal design indicate a good fit with R2 of 0.916 and 0.906, respectively. Parametric studies based on the two models showed that ash yield and CO emission reduced with increased combustion temperature and increased fraction of PKS within the temperature range of 800-1000 °C. These results indicated that both ANN and D-optimal can be deployed to select mixture with minimal ash yield and CO emission.

Nonlinear Neural Network Based Forecasting Model for Predicting COVID-19 Cases.

The recent COVID-19 outbreak has severely affected people around the world. There is a need of an efficient decision making tool to improve awareness about the spread of COVID-19 infections among the common public. An accurate and reliable neural network based tool for predicting confirmed, recovered and death cases of COVID-19 can be very helpful to the health consultants for taking appropriate actions to control the outbreak. This paper proposes a novel Nonlinear Autoregressive (NAR) Neural Network Time Series (NAR-NNTS) model for forecasting COVID-19 cases. This NAR-NNTS model is trained with Scaled Conjugate Gradient (SCG), Levenberg Marquardt (LM) and Bayesian Regularization (BR) training algorithms. The performance of the proposed model has been compared by using Root Mean Square Error (RMSE), Mean Square Error (MSE) and correlation co-efficient i.e. R-value. The results show that NAR-NNTS model trained with LM training algorithm performs better than other models for COVID-19 epidemiological data prediction.

Performance Analysis of Traffic Congestion Using Designated Neural Network Training Algorithms

A lot of neural network training algorithms on prediction exist and these algorithms are being used by researchers to solve evaluation, forecasting, clustering, function approximation etc. problems in traffic volume congestion. This study is aimed at analysing the performance of traffic congestion using some designated neural network training algorithms on traffic flow in some selected corridors within Akure, Ondo state, Nigeria. The selected corridors were Oba Adesida road, Oyemekun road and Oke Ijebu road all in Akure. The traffic flow data were collected manually with the help of field observers who monitored and record traffic movement along the corridors. To accomplish this, three common training algorithms were selected to train the traffic flow data. The data were trained using Bayesian Regularization (BR), Scaled Conjugate Gradient (SCG) and Levenberg-Marquardt (LM) training algorithms. The outputs/performances of these training functions were evaluated by using the Mean Square Error (MSE) and Coefficient of Regression (R) to find the best training algorithms. The results show that, the Bayesian regularization algorithm, performs better with MSE of 2.37e-13 and R of 0.9999 than SCG and LM algorithms.

Improving the predictive accuracy of artificial neural network (ANN) approach in a mild steel turning operation

Improving and increasing metal cutting process efficiency requires an in-depth knowledge of the cutting process, as well as the development/application of an efficient modeling technique, capable of forming the core tools for planning, scheduling, and integration of the machining processes. This study evaluates the application of Levenberg Marquardt (LM) and Scaled Conjugate Gradient (SCG) training algorithms of artificial neural networks in the modeling and prediction of metal removal rate (MRR) and average surface roughness (Ra) in the turning of cylindrical mild steel. The neural network models were trained using a set of experimental data consisting of the input parameters: cutting speed, depth of cut, and feed rate. Based on the graphical and statistical results, the LM offered the best accuracy in terms of the model fitting, RMSE and R2 obtained. For the LM algorithms, the least RMSE values of 1.7809 × 10−1, 2.6805 × 10−1, and 1.5815 × 10−1 were obtained for the training, validation, and test data respectively, and for the SCG algorithm, least RMSE values of 1.7809 × 10−1, 2.6805 × 10−1, and 1.5815 × 10−1 were recorded. Comparing both algorithms, the Levenberg Marquardt algorithm recorded the least average in terms of the number of the epoch, and RMSE value and at the same time the best performance in terms of good fitting.

Discrimination of Two-Class Motor Imagery in a fNIRS Based Brain Computer Interface.

The purpose of this study was to discriminate between left- and right-hand motor imagery tasks. We recorded the brain signals from two participants using a fNIRS system and compared different feature extraction (mean, peak, minimum, skewness and kurtosis) and classification techniques (linear (LDA) and quadratic discriminant analysis (QDA), support vector machine (SVM), logistic regression, K-nearest-neighbor (KNN), and neural networks with Levenberg-Marquardt (LMA), Bayesian Regularization (BRANN) and Scaled Conjugate Gradient (SCGA) training algorithms). The results showed poor classification accuracies (<; 58%) when skewness and kurtosis were used. When mean, peak, and minimum were used as features, QDA, SVM and KNN produced higher classification accuracies relative to LDA and logistic regression. Overall, BRANN led to the highest accuracies (>98%) when mean, peak and minimum were used as features.

Artificial Neural Networks Based Solar Radiation Estimation using Backpropagation Algorithm

Being at the cutting edge for a long time, solar energy has found several applications in various areas. Optimal harvesting of solar energy is one of the thrust areas of the researchers and developers in 21 st century. Solar energy optimization solely depends on radiation received by the solar panels. Radiation is measured by various devices and it may be estimated by various estimation models. Solar energy needs to be estimated well in advance if the system to be designed has to be completely dependent on solar energy. This is a very challenging job since solar radiation depends on several parameters such as location changes and seasonal changes. Artificial Neural Network is one of the most preferred technique for the researchers in prediction related cases. This paper proposes a methodology to estimate solar radiation using feed forward back propagation neural network. A three-layer neural network is used here with one hidden layer. The data of 14 stations of Uttar Pradesh, India are obtained data from FAO, UN and it further divided into three sets of ‘Training’, Validation’ and ‘Testing’. This study is based on nine input parameters and one output parameter. Feed forward back propagation is used here to estimate solar radiation. The proposed model is validated for Levenberg-Marquardt, Bayesian Regularization, and Scaled Conjugate Gradient training algorithms. The obtained results of Mean Square Error (MSE), Regression Values (R), Slope Values (m) and Intercept Values (c) in all three cases are justifying the suitability of the proposed model.

DWT-based electromyogram signal classification using maximum likelihood-estimated features for neurodiagnostic applications

Automated diagnosis of neuromuscular disorders such as myopathy and neuropathy can be done by measuring and analyzing the nonlinear and non-stationary trends in electromyogram (EMG) signals. This paper introduces a new automated diagnostic approach with the combination of discrete wavelet transform (DWT) and maximum likelihood estimation (ML-estimation) for analysis and classification of EMG signals into healthy, myopathy, or neuropathy classes. DWT decomposes the considered EMG signals into frequency sub-bands, and ML-estimation is used for extracting the features from the discrete wavelet sub-bands. Subsequently, obtained feature vectors are used for classifying the EMG signals using multilayer perceptron neural network (MLPNN). Comparative analysis of various performance measures has been performed with all the 71 wavelets of DWT and 13 training algorithms of MLPNN. Results are promising with specificity of 83.33%, sensitivity (myopathy) of 91.66%, sensitivity (neuropathy) of 83.33%, and total classification accuracy of 86.5%, which are obtained using the combination of Daubechies-11 wavelet and Fletcher–Reeves conjugate gradient training algorithm.

Artificial Intelligence Modelling Approach for the Prediction of CO-Rich Hydrogen Production Rate from Methane Dry Reforming

This study investigates the applicability of the Leven–Marquardt algorithm, Bayesian regularization, and a scaled conjugate gradient algorithm as training algorithms for an artificial neural network (ANN) predictively modeling the rate of CO and H2 production by methane dry reforming over a Co/Pr2O3 catalyst. The dataset employed for the ANN modeling was obtained using a central composite experimental design. The input parameters consisted of CH4 partial pressure, CO2 partial pressure, and reaction temperature, while the target parameters included the rate of CO and H2 production. A neural network architecture of 3 13 2, 3 15 2, and 3 15 2 representing the input layer, hidden neuron layer, and target (output) layer were employed for the Leven–Marquardt, Bayesian regularization, and scaled conjugate gradient training algorithms, respectively. The ANN training with each of the algorithms resulted in an accurate prediction of the rate of CO and H2 production. The best prediction was, however, obtained using the Bayesian regularization algorithm with the lowest standard error of estimates (SEE). The high values of coefficient of determination (R2 &gt; 0.9) obtained from the parity plots are an indication that the predicted rates of CO and H2 production were strongly correlated with the observed values.

Generalized Dai-Yuan conjugate gradient algorithm for training multi-layer feed-forward neural networks

In this paper, we will present different type of CG algorithms depending on Peary conjugacy condition. The new conjugate gradient training (GDY) algorithm using to train MFNNs and prove it's descent property and global convergence for it and then we tested the behavior of this algorithm in the training of artificial neural networks and compared it with known algorithms in this field through two types of issues.

Performance Analysis of Multilayer Perceptron Neural Network Models in Week-Ahead Rainfall Forecasting

Multilayer perceptron neural network (MLPNN) is considered as one of the most efficient forecasting techniques which can be implemented for the prediction of weather occurrence. As with any machine learning implementation, the challenge on the utilization of MLPNN in rainfall forecasting lies in the development and evaluation of MLPNN models which delivers optimal forecasting performance. This research conducted performance analysis of MLPNN models through data preparation, model designing, and model evaluation in order to determine which parameters are the best-fit configurations for MLPNN model implementation in rainfall forecasting. During rainfall data preparation, imputation process and spatial correlation evaluation of weather variables from various weather stations showed that the geographical location of the chosen weather stations did not have a direct correlation between stations with respect to rainfall behavior leading to the decision of utilizing the weather station having the most complete weather data to be fed in the MLPNN. By conducting performance analysis of MLPNN models with different combinations of training algorithms, activation functions, learning rate, and momentum, it was found out that MLPNN model having 100 hidden neurons with Scaled Conjugate Gradient training algorithm and Sigmoid activation function delivered the lowest RMSE of 0.031537 while another MLPNN model having the same number of hidden neurons, the same activation function but Resilient Propagation as training algorithm had the lowest MAE of 0.0209. The results of this research showed that performance analysis of MLPNN models is a crucial process in model implementation of MLPNN for week-ahead rainfall forecasting.

A New Conjugate Gradient Method with Smoothing $$L_{1/2} $$ L 1 / 2 Regularization Based on a Modified Secant Equation for Training Neural Networks

Proposed in this paper is a new conjugate gradient method with smoothing $$L_{1/2} $$ regularization based on a modified secant equation for training neural networks, where a descent search direction is generated by selecting an adaptive learning rate based on the strong Wolfe conditions. Two adaptive parameters are introduced such that the new training method possesses both quasi-Newton property and sufficient descent property. As shown in the numerical experiments for five benchmark classification problems from UCI repository, compared with the other conjugate gradient training algorithms, the new training algorithm has roughly the same or even better learning capacity, but significantly better generalization capacity and network sparsity. Under mild assumptions, a global convergence result of the proposed training method is also proved.

Modeling of Groundwater Resources Heavy Metals Concentration Using Soft Computing Methods: Application of Different Types of Artificial Neural Networks

Nowadays, groundwater resources play a vital role as a source of drinking water in arid and semiarid regions and forecasting of pollutants content in these resources is very important. Therefore, this study aimed to compare two soft computing methods for modeling Cd, Pb and Zn concentration in groundwater resources of Asadabad Plain, Western Iran. The relative accuracy of several soft computing models, namely multi-layer perceptron (MLP) and radial basis function (RBF) for forecasting of heavy metals concentration have been investigated. In addition, Levenberg-Marquardt, gradient descent and conjugate gradient training algorithms were utilized for the MLP models. The ANN models for this study were developed using MATLAB R 2014 Software program. The MLP performs better than the other models for heavy metals concentration estimation. The simulation results revealed that MLP model was able to model heavy metals concentration in groundwater resources favorably. It generally is effectively utilized in environmental applications and in the water quality estimations. In addition, out of three algorithms, Levenberg-Marquardt was better than the others were. This study proposed soft computing modeling techniques for the prediction and estimation of heavy metals concentration in groundwater resources of Asadabad Plain. Based on collected data from the plain, MLP and RBF models were developed for each heavy metal. MLP can be utilized effectively in applications of prediction of heavy metals concentration in groundwater resources of Asadabad Plain.

Determination of New Bio Signal and Tests Alternative to Verbal Pain Scale for Diagnosing Fibromyalgia Syndrome

In this study; values obtained through the analysis of blood samples, taken under laboratory conditions, from patients diagnosed with fibromyalgia syndrome and healthy subjects and the sympathetic skin response parameters were used. With the aim of classifying verbal pain scale, which is one of the psychological test scores used for fibromyalgia syndrome diagnosis; relation between the sympathetic skin response effect on other test data and the verbal pain scale were reviewed by using different conditions of available data. Within this framework, three different algorithms were used for classification with high accuracy rates. These algorithms are: Multi-Layer Feed-Forward Neural Networks, Probabilistic Neural Network and Radial Basis Function Neural Network. For Multi-Layer Feed-Forward Neural Networks classification algorithm, classification was done with three different training algorithms, Levenberg-Marquardt back propagation, Resilient back propagation and the Scaled conjugate gradient back propagation and the results were compared elaborately. Based on the results, by using all variables the following accuracy rates were obtained: 68.2% accuracy with Levenberg-Marquardt training algorithm, 77.3% accuracy with the Resilient back propagation training algorithm, and 68.18% accuracy with the Scaled conjugate gradient training algorithm. These success rates show that there is a relationship between verbal pain scale, sympathetic skin response and other test data.