Artificial Neural Network Training Method Research Articles

The Comparison of the Multi-Layer Artificial Neural Network Training Methods in Terms of the Predictive Quality of the Coefficient of Friction of 1.0338 (DC04) Steel Sheet.

Friction is one of the main phenomena accompanying sheet metal forming methods, affecting the surface quality of products and the formability of the sheet metal. The most basic and cheapest way to reduce friction is to use lubricants, which should ensure the highest lubrication efficiency and at the same time be environmentally friendly. Due to the trend towards sustainable production, vegetable oils have been used in research as an alternative to petroleum-based lubricants. The analysis of friction in sheet metal forming requires an appropriate tribotester simulating the friction conditions in a specific area of the sheet metal being formed. Research has used a special strip drawing tribometer, enabling the determination the value of the coefficient of friction in the blankholder zone in the deep drawing process. Quantitative analysis of the friction phenomenon is necessary at the stage of designing the technological process and selecting technological parameters, including blankholder pressure. This article presents the results of friction testing of 1.0338 (DC04) steel sheets using a strip drawing test. The experimental tests involved pulling a strip of sheet metal between two countersamples with a rounded surface. The tests were carried out on countersamples with different levels of roughness for the range of contact pressures occurring in the blankholder zone in the deep drawing process (1.7-5 MPa). The values of the coefficient of friction determined under dry friction conditions were compared with the results for edible (corn, sunflower and rapeseed) and non-edible (Moringa, Karanja) vegetable lubricants. The tested oils are the most commonly used vegetable-based biolubricants in metal forming operations. Multi-layer artificial neural networks were used to determine the relationship between the value of the contact pressure, the roughness of the countersamples, the oil viscosity and density, and the value of the coefficient of friction. Rapeseed oil provided the best lubrication efficiency during friction testing for all of the tested samples, with an average surface roughness of Sa 0.44-1.34 μm. At the same time, as the roughness of the countersamples increased, a decrease in lubrication efficiency was observed. The lowest root mean squared error value was observed for the MLP-4-8-1 network trained with the quasi-Newton algorithm. Most of the analysed networks with different architectures trained using the various algorithms showed that the kinematic viscosity of the oil was the most important aspect in assessing the friction of the sheets tested. The influence of kinematic viscosity on the value of the coefficient of friction is strongly dependent on the surface roughness of the countersamples.

Context-Aware Clustering and the Optimized Whale Optimization Algorithm: An Effective Predictive Model for the Smart Grid

For customers to participate in key peak pricing, period-of-use fees, and individualized responsiveness to demand programmes taken from multi-dimensional data flows, energy use projection and analysis must be done well. However, it is a difficult study topic to ascertain the knowledge of use of electricity as recorded in the electricity records' Multi-Dimensional Data Streams (MDDS). Context-Aware Clustering (CAC) and the Optimized Whale Optimization Algorithm were suggested by researchers as a fresh power usage knowledge finding model from the multi-dimensional data streams (MDDS) to resolve issue (OWOA). The proposed CAC-OWOA framework first performs the data cleaning to handle the noisy and null elements. The predictive features are extracted from the novel context-aware group formation algorithm using the statistical context parameters from the pre-processed MDDS electricity logs. To perform the energy consumption prediction, researchers have proposed the novel Artificial Neural Network (ANN) predictive algorithm using the bio-inspired optimization algorithm called OWOA. The OWOA is the modified algorithm of the existing WOA to overcome the problems of slow convergence speed and easily falling into the local optimal solutions. The ANN training method is used in conjunction with the suggested bio-inspired OWOA algorithm to lower error rates and boost overall prediction accuracy. The efficiency of the CAC-OWOA framework is evaluated using the publicly available smart grid electricity consumption logs. The experimental results demonstrate the effectiveness of the CAC-OWOA framework in terms of forecasting accuracy, precision, recall, and duration when compared to underlying approaches.

Uma proposta de arquitetura da informação aplicada ao processamento de linguagem natural: contribuições da Ciência da Informação no pré-processamento de dados para treinamento e aprendizado de redes neurais artificiais

Introduction:Natural Language Processing through artificial neural networks has gaps that can be addressed by Information Science through Information Architecture. Objective:To present Information Science contributions on Knowledge Organization applied to artificial neural networks training methods, positioning it as an active body of knowledge in artificial intelligence problems. Methodology:A three-leveled analysis path (metaphysical, scientific, and technological) is adopted to guide and ground the study. On metaphysical level, current development stage of natural language processing techniques is verified and analyzed. On scientific findings, a five-step procedure is proposed which aims to design, analyze, and prepare information spaces for artificial neural networks training and learning methods, fulfilling gaps identified by authors focused on Computer Science implementations. On technological implementation, the five-step procedure is applied to 3 datasets formed by texts from 16 scientific knowledge areas, as an evaluation basis. Results:Results obtained through pre-processed data and raw data where compared, showing great potential in developing a structuredmethod of Multimodal Information Architecture that provide instruments able to organize data used as test and learning samples in artificial neural networks. Conclusion:This method could place Information Science as a producer of data pre-processing solutions, replacing its current role as consumer of prefabricated solutions made by Computer Science.

Statistical Modeling and Optimization of Process Parameters for 2,4-Dichlorophenoxyacetic Acid Removal by Using AC/PDMAEMA Hydrogel Adsorbent: Comparison of Different RSM Designs and ANN Training Methods

In this study, the response surface methodology (RSM) and artificial neural network (ANN) were employed to study the adsorption process of 2,4-dichlorophenoxyacetic acid (2,4-D) by using modified hydrogel, i.e., activated carbon poly(dimethylaminoethyl methacrylate) (AC/PDMAEMA hydrogel). The effect of pH, the initial concentration of 2,4-D and the activated carbon content on the removal of 2,4-D and adsorption capacity were investigated through the face-centered composite design (FCCD), optimal design and two-level factorial design. The response surface plot suggested that higher removal of 2,4-D and adsorption capacity could be achieved at the higher initial concentration of 2,4-D and lower pH and activated carbon content. The modeling and optimization for the adsorption process of 2,4-D were also carried out by different design methods of RSM and different training methods of ANN. It was found that among the three design methods of RSM, the optimal design has the highest accuracy for the prediction of 2,4-D removal and adsorption capacity (R2 = 0.9958 and R2 = 0.9998, respectively). The numerical optimization of the optimal design found that the maximum removal of 2,4-D and adsorption capacity of 65.01% and 65.29 mg/g, respectively, were obtained at a pH of 3, initial concentration of 2,4-D of 94.52 mg/L and 2.5 wt% of activated carbon. Apart from the optimization of process parameters, the neural network architecture was also optimized by trial and error with different numbers of hidden neurons in the layers to obtain the best performance of the response. The optimization of the neural network was performed with different training methods. It was found that among the three training methods of the ANN model, the Bayesian Regularization method had the highest R2 and lowest mean square error (MSE) with the optimum network architecture of 3:9:2. The optimum condition obtained from RSM was also simulated with the optimized neural network architecture to validate the responses and adequacy of the RSM model.

Analysis of artificial neural networks training models for airfare price prediction

Air transport is playing an increasing role in the world economy every year. This is facilitated by technological development and the latest developments in the aviation industry, globalization. This paper provides an overview of artificial neural network training methods for airfare predicting. The articles for 2017-2019 were analyzed in order to determine the model with the most accurate prediction. The researchers conducted research on open data collected by themselves and set themselves the goal of creating a model that would advise a user the best time to buy a ticket when the price would be the lowest. The review of the papers by similar themes revealed that the Bagging Regression Tree model has the highest results with an accuracy of 88% and the random forest method has an accuracy of 87%. Civil aviation plays an important role in the economy of each country. Aviation is the best way to cover long distances in comfort in the shortest time. Airlines offer customers a variety of opportunities to travel both within the country and abroad. The main problem of interaction between airlines and customers is the airfare: the former want to sell more at the higher price, and the latter want to buy cheaper. Therefore, companies use their own private algorithms for dynamic pricing and constantly monitor the market situation, responsive to changes in demand and the actions of competitors. This behavior allows them to achieve a balance between the desires of airlines and customers. Scientists are trying to invent a way to predict airfare so that air travelers can buy them at the lowest price. The results of the work in this area provide general rules for the best purchase. For example, according to the article (Udachny, 2016) thebest day to buy a ticket by expedia.com for a domestic flight on the United States is Sunday, and the best period is 57 days before departure. This article provides an overview of the works, the authors of which compared the models of machine learning. Achievements in this area are limited to direct flights of a certain domestic market (USA, India) and 88% accuracy of the forecast (Tziridis et al., 2017). The Bagging Regression Tree model described in the article (Tziridis et al., 2017) can be considered the best result. This trained model can make predictions based only on two parameters: the number of free cargo and the number of days left before departure and has an accuracy of 88%.

A Holistic Model for Recognition of Handwritten Arabic Text Based on the Local Binary Pattern Technique

<p class="0abstract">In this paper, we introduce a multi-stage offline holistic handwritten Arabic text recognition model using the Local Binary Pattern (LBP) technique and two machine-learning approaches; Support Vector Machines (SVM) and Artificial Neural Network (ANN). In this model, the LBP method is utilized for extracting the global text features without text segmentation. The suggested model was tested and utilized on version II of the IFN/ENIT database applying the polynomial, linear, and Gaussian SVM and ANN classifiers. Performance of the ANN was assessed using the Levenberg-Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG) training methods. The classification outputs of the herein suggested model were compared and verified with the results obtained from two benchmark Arabic text recognition models (ATRSs) that are based on the Discrete Cosine Transform (DCT) and Principal Component Analysis (PCA) methods using various normalization sizes of images of Arabic text. The classification outcomes of the suggested model are promising and better than the outcomes of the examined benchmarks models. The best classification accuracies of the suggested model (97.46% and 94.92%) are obtained using the polynomial SVM classifier and the BR ANN training methods, respectively.</p>

Resilient training of neural network classifiers with approximate computing techniques for hardware‐optimised implementations

As Machine Learning applications increase the demand for optimised implementations in both embedded and high-end processing platforms, the industry and research community have been responding with different approaches to implement these solutions. This work presents approximations to arithmetic operations and mathematical functions that, associated with a customised adaptive artificial neural networks training method, based on RMSProp, provide reliable and efficient implementations of classifiers. The proposed solution does not rely on mixed operations with higher precision or complex rounding methods that are commonly applied. The intention of this work is not to find the optimal simplifications for specific deep learning problems but to present an optimised framework that can be used as reliably as one implemented with precise operations, standard training algorithms and the same network structures and hyper-parameters. By simplifying the ‘half-precision’ floating point format and approximating exponentiation and square root operations, the authors’ work drastically reduces the field programmable gate array implementation complexity (e.g. −43 and −57% in two of the component resources). The reciprocal square root approximation is so simple it could be implemented only with combination logic. In a full software implementation for a mixed-precision platform, only two of the approximations compensate the processing overhead of precision conversions.

Determining optimal artificial neural network training method in predicting the performance and emission parameters of a biodiesel-fueled diesel generator

Artificial neural network (ANN) methods were employed and suggested in modeling the emissions and performance of a diesel generator fueled with waste cooking oil derived biodiesel during steady-state operation. These papers are generally built on determining optimal network structure, but the modelling accuracy of an ANN is also highly dependent on employed training method. In modeling, operating conditions and fuel blend ratio were used as the inputs while the performance and emission parameters were the outputs. The modeling results obtained by conventional ANNs that were trained by back propagation (BP) learning algorithm, radial basis function (RBF), and extreme learning machine (ELM) were compared with experimental results and each other. The accuracy of the estimations by ELM was above 95% for all the output parameters except for specific fuel consumption and thermal efficiency. Moreover, ELM performed better than BP and RBF with lower mean relative error (MRE) in case where the emissions were estimated. The ELM provided correlation coefficients of 0.987, 0.950 and 0.996 for unburned hydrocarbons (HCs), nitrogen oxides (NOx) and smoke opacity (SO), respectively, while for BP, they were 0.973, 0.818, 0.993, and for RBF, 0.975, 0.640 and 0.981. The most suitable training function for each emission and performance parameters of diesel generator was determined based on obtained accuracies.

Multivariate computational analysis of biosensor's data for improved CD64 quantification for sepsis diagnosis.

Sepsis, as a leading cause of death worldwide, relies on systemic inflammatory response syndrome (SIRS) criteria for its diagnosis. SIRS is highly non-specific as it relies on monitoring of patients' vitals for sepsis diagnosis, which are known to change with many confounding factors. Changes in leukocyte counts and CD64 expression levels are known specific biomarkers of pro-inflammatory host response at the onset of sepsis. Recently, we have developed a biosensor chip that can enumerate the leukocyte counts and quantify the neutrophil CD64 expression levels from a drop of blood. We were able to show improved sepsis diagnosis and prognosis in clinical studies by measuring these parameters during different times of the patients' stay in hospital. In this paper, we investigated the rate of cell capture with CD64 expression levels and used this in a multivariate computational model using artificial neural networks (ANNs) and showed improved accuracy of quantifying CD64 expression levels from the biosensor (n = 106 whole blood experiments). We found a high coefficient of determination and low error between biosensor- and flow cytometry-based neutrophil CD64 expression levels using multiple ANN training methods in comparison to those of univariate regression commonly employed. This approach can find many applications in biosensor data analytics by utilizing multiple features of the biosensor's data for output determination.

Prediction of Wave-induced Liquefaction using Artificial Neural Network and Wide Genetic Algorithm


 The hassle of analytical and numerical solution for liquefaction modeling, repetitive laboratory testing and expensive field observations, have opened opportunities to develop simple, practical, inexpensive and valid prediction of wave-induced liquefaction. In this study, Artificial Neural Network (ANN) regression modeling is used to predict the depth of liquefaction. Despite of using Back Propagation (BP) to train ANN, a modified Genetic Algorithm (called as Wide GA, WGA) is used as ANN training method to improve ANN prediction accuracy and to overcome BP weaknesses such as premature convergence and local optimum. WGA also aim to avoid conventional GA weaknesses such as low population diversity and narrow search coverage. Key WGA operations are Wide Tournament Selection, Multi-Parent BLX-? Crossover, Aggregate Mate Pool Mutation and Direct Fresh Mutation-Crossover. ANN prediction accuracy measured by Median APE (MdAPE). Global optimum solution of WGA is best ANN connections weights configuration with smallest MdAPE.

Training Neural Networks with Krill Herd Algorithm

In recent times, several new metaheuristic algorithms based on natural phenomena have been made available to researchers. One of these is that of the Krill Herd Algorithm (KHA) procedure. It contains many interesting mechanisms. The purpose of this article is to compare the KHA optimization algorithm used for learning an artificial neural network (ANN), with other heuristic methods and with more conventional procedures. The proposed ANN training method has been verified for the classification task. For that purpose benchmark examples drawn from the UCI Machine Learning Repository were employed with Classification Error and Sum of Square Errors being used as evaluation criteria. It has been concluded that the application of KHA offers promising performance--both in terms of aforementioned metrics, as well as time needed for ANN training.

PSO-BP Combined Artificial Neural Network Method Research

BP artificial neural network(ANN) based on gradient algorithm method is widely applied, but because the error surface of object function is very complex and the choose of initial value effects network training results, convergence rate is slow and local minimum is likely to fall into. Particle swarm optimization(PSO) algorithm has better global searching ability to get rid the puzzles of falling into local minimum. By adequately studying on the two algorithms’ characteristics, a new type of combined ANN training method is put forward, and PSO-BP ann model is successfully built.

Degenerated simplex search method to optimize neural network error function

PurposeThe purpose of this paper is to present a degenerated simplex search method to optimize neural network error function. By repeatedly reflecting and expanding a simplex, the centroid property of the simplex changes the location of the simplex vertices. The proposed algorithm selects the location of the centroid of a simplex as the possible minimum point of an artificial neural network (ANN) error function. The algorithm continually changes the shape of the simplex to move multiple directions in error function space. Each movement of the simplex in search space generates local minimum. Simulating the simplex geometry, the algorithm generates random vertices to train ANN error function. It is easy to solve problems in lower dimension. The algorithm is reliable and locates minimum function value at the early stage of training. It is appropriate for classification, forecasting and optimization problems.Design/methodology/approachAdding more neurons in ANN structure, the terrain of the error function becomes complex and the Hessian matrix of the error function tends to be positive semi‐definite. As a result, derivative based training method faces convergence difficulty. If the error function contains several local minimum or if the error surface is almost flat, then the algorithm faces convergence difficulty. The proposed algorithm is an alternate method in such case. This paper presents a non‐degenerate simplex training algorithm. It improves convergence by maintaining irregular shape of the simplex geometry during degenerated stage. A randomized simplex geometry is introduced to maintain irregular contour of a degenerated simplex during training.FindingsSimulation results show that the new search is efficient and improves the function convergence. Classification and statistical time series problems in higher dimensions are solved. Experimental results show that the new algorithm (degenerated simplex algorithm, DSA) works better than the random simplex algorithm (RSM) and back propagation training method (BPM). Experimental results confirm algorithm's robust performance.Research limitations/implicationsThe algorithm is expected to face convergence complexity for optimization problems in higher dimensions. Good quality suboptimal solution is available at the early stage of training and the locally optimized function value is not far off the global optimal solution, determined by the algorithm.Practical implicationsTraditional simplex faces convergence difficulty to train ANN error function since during training simplex can't maintain irregular shape to avoid degeneracy. Simplex size becomes extremely small. Hence convergence difficulty is common. Steps are taken to redefine simplex so that the algorithm avoids the local minimum. The proposed ANN training method is derivative free. There is no demand for first order or second order derivative information hence making it simple to train ANN error function.Originality/valueThe algorithm optimizes ANN error function, when the Hessian matrix of error function is ill conditioned. Since no derivative information is necessary, the algorithm is appealing for instances where it is hard to find derivative information. It is robust and is considered a benchmark algorithm for unknown optimization problems.

Fault Distance Measurement of Series Compensated Lines using ANN

This paper presents the effects on the distance protection scheme used by the transmission lines protection system when they are compensated in series by Thyristor Controlled Series Capacitor (TCSC). The usage of TCSC introduces problems to distance protection schemes as there is a change in the measurement of apparent impedance observed by the respective relay. The change seen by the relay measurement has to be corrected for accurate operation of the particular relay, for which we take the help of Artificial Neural Networks which is extensively tested for various test conditions in MATLAB Simulation environment. The relay operation accuracy used for distance protection is improved by using the Artificial Neural Networks training methods and it also shows the distance at which the faults has occurred.

A multitarget training method for artificial neural network with application to computer-aided diagnosis.

The authors propose a new training method for artificial neural networks (ANNs) in two-class classification tasks such as classifying breast lesions on a mammogram as malignant or benign. Whereas the conventional binary training method uses binary training target values based on the diagnostic truth of a lesion being malignant or benign, the authors use multiple training target values based on more detailed histological diagnosis that presumably are related to the posterior probability of a lesion being malignant. The authors performed Monte Carlo simulation studies in which training target values were assigned based on posterior probability, and they also performed a mammography study in which training target values were assigned according to histological subtypes. These studies showed that the multitarget training method produced less variability in the ANN outputs than the binary training method. The simulation studies also showed that except for when the number of training cases was extremely large, the multitarget training method produced improved overall classification performance over the binary training method. Therefore, the multitarget ANN training method is potentially useful for ANN applications in computer-aided diagnosis of breast cancer.

Generation and simplification of Artificial Neural Networks by means of Genetic Programming

The development of Artificial Neural Networks (ANNs) is traditionally a slow process in which human experts are needed to experiment on different architectural procedures until they find the one that presents the correct results that solve a specific problem. This work describes a new technique that uses Genetic Programming (GP) in order to automatically develop simple ANNs, with a low number of neurons and connections. Experiments have been carried out in order to measure the behavior of the system and also to compare the results obtained using other ANN generation and training methods with evolutionary computation (EC) tools. The obtained results are, in the worst case, at least comparable to existing techniques and, in many cases, substantially better. As explained herein, the system has other important features such as variable discrimination, which provides new information on the problems to be solved.

Performance comparison of neural network training methods based on wavelet packet transform for classification of five mental tasks

In this study, performances comparison to discriminate five mental states of five artificial neural network (ANN) training methods were investigated. Wavelet Packet Transform (WPT) was used for feature extraction of the relevant frequency bands from raw electroencephalogram (EEG) signals. The five ANN training methods used were (a) Gradient Descent Back Propagation (b) Levenberg-Marquardt (c) Resilient Back Propagation (d) Conjugate Learning Gradient Back Propagation and (e) Gradient Descent Back Propagation with movementum.

A novel Artificial Neural Network training method combined with Quantum Computational Multi-Agent System theory

Artificial Neural Networks (ANNs) are powerful tools that can be used to model and investigate various complex and non-linear phenomena. In this study, we construct a new ANN, which is based on Multi-Agent System (MAS) theory and quantum computing algorithm. All nodes in this new ANN are presented as Quantum Computational (QC) agents, and these agents have learning ability. A novel ANN training method was proposed via implementing QCMAS reinforcement learning. This new ANN has powerful parallel-work ability and its training time is shorter than classic algorithm. Experiment results show that this method is effective.

A neural network learning algorithm of chemical process modeling based on the extended Kalman filter

The extended Kalman filtering (EKF) algorithm instead of the error back-propagation (BP) algorithm is used to train artificial neural networks (ANNs) for chemical process modeling. The basic idea is, by modifying the EKF gain, to prevent overfitting or filtering divergence phenomenon caused by outliers in the training samples. The EKF-based ANNs training method proposed is also applied to estimate the conversion rate in the polyacrylonitrile production process. Numerical simulations show that the modified EKF algorithm is superior to the BP algorithm in resisting noise and outliers, as well as generalization.

Particle swarm algorithm trained neural network for QSAR studies of inhibitors of platelet-derived growth factor receptor phosphorylation

The multilayer feed-forward artificial neural network (ANN) has been widely used in QSAR studying. Back-propagation algorithm (BP) and the use of evolutionary search as an ANN training method has some limitations associated with overfitting, local optimum problems and slow convergence rate. In this paper, particle swarm optimization (PSO) as a relatively new optimization technique has been used in ANN training. Compared to ANN trained by BP algorithm and evolutionary search, ANN training by PSO algorithm (PSONN) show satisfactory performance, converges quickly towards the optimal position and can avoid overfitting in some extent. The PSONN has been testified by using in QSAR modeling for inhibitory activity of 4-[4-( N-substituted (thio) carbamoyl)-1-piperazinyl]-6,7-dimethoxyquinazoline derivatives.