Gradient Descent Back Propagation Algorithm Research Articles

Interval Type-2 Mutual Subsethood Cauchy Fuzzy Neural Inference System (IT2MSCFuNIS)

An interval type-2 (IT2) mutual subsethood Cauchy fuzzy neural inference system has been proposed in this paper. The network architecture consists of 3-layers with all connection weights being IT2 Cauchy fuzzy membership functions (CMFs). The crisp inputs to the system are fuzzified into IT2CMFs with fixed centers and uncertain spreads. The hidden layer represents the rule-based knowledge. The firing degree of the antecedent part of each rule at the hidden layer is computed by aggregating the product of the mutual subsethood similarity measures between the inputs and the connection weights. A volume defuzzification is used to compute the numeric output. A gradient descent back-propagation algorithm is used to train the model. The novelty of the proposed model is threefold. First, is enriching the theory of the mutual subsethood fuzzy neural models by adopting the Cauchy membership function (CMF) as another powerful fuzzy basis function (FBF) rather than the classical choice of Gaussian fuzzy membership functions (GMFs). Second, is the success of computing the mutual subsethood similarity measure between the IT2CMFs and all the model parameters’ updating equations in analytic closed-form formulas, not numerically or approximately. Third, is the ability to extract the type-1 (T1) mutual subsethood Cauchy fuzzy neural inference system (T1MSCFuNIS) with all its analytic closed-form formulas directly as a special case from the general formulas of IT2MSCFuNIS model. Such a novelty makes the proposed model a concrete and effective development of the theory of mutual subsethood fuzzy neural models. Both IT2MSCFuNIS and T1MSCFuNIS models have been tested using different examples from the domains of function approximation, classification, and prediction. The results ensure the efficacy of both models compared with other models reported in the literature.

The feasibility of designing a back propagation neural network to predict the levitation force of high-temperature superconducting magnetic levitation

The levitation force between the superconductor and the magnet is highly nonlinear and affected by the coupling of multiple factors, which brings many obstacles to research and application. In addition to experimental methods and finite element simulations, the booming artificial neural network (ANN) which is adept at continuous nonlinear fitting may provide another solution to predict the levitation force. And this topic has not been deeply investigated so far. Therefore, this study aims to apply the ANN to predict the levitation force, and a typical neural network applied with the back propagation (BP) is adopted. The data set with 2399 pieces of data considers nine input factors and one force output, which was experimentally obtained by several test devices. The pre-process of the data set contains cleaning, balancing, one-hot encoding (for the discrete classified variable), normalization (for the continuous variable) and randomization. A classical perception with three layers (input, hidden and output layer) is applied in this paper. And the gradient descent back propagation algorithm reduces the error by iteration. Through the assessment and evaluation of the network, a great prediction accuracy could achieve. The prediction results could well illustrate the features of force (nonlinear, hysteresis, external field dependence and type difference between the bulk and stack), which confirm the feasibility of using a BP neural network to predict the levitation force. Furthermore, the performance of the neural network is determined by the data set, especially the uniformity and balance among factors in the set. Moreover, the huge gap in the quantity of data between factors disturbs the network to make a comprehensive judgment, and in this situation, the binary one-hot encoding of the small quantity and discrete data factor is efficient, instead of the actual value of the factor, the one-hot encoded data only represent the category. Moreover, a label encoder method is adopted to distinguish the decent and ascend (decent = 1, ascent = 0) for the force hysteresis.

Artificial Neural Network Context-Aware Reasoning Engine Model for Predicting User-Ranked Activities

The choice of users’ activity in a context-aware environment depends on users’ preferences and background. Users tend to rank concurrent activities and select their preferred activity. Researchers and developers of context-aware applications have sought various mechanisms to implement context reasoning engines. Recent implementations use Artificial Neural Networks (ANN) and other machine learning techniques to develop a context-aware reasoning engine to predict users’ activities. However, the complexities of these mechanisms overwhelm the processing capabilities and storage capacity of mobile devices. The study models a context-aware reasoning engine using a multi-layered perceptron with a gradient descent back-propagation algorithm to predict activity from user-ranked activities using a stochastic learning mode with a constant learning rate. The work deduced that working with specific rules in training a neural network is not always applicable. Training a network without approximation of neuron’s output to the nearest whole number increases the accuracy level of the network at the end of the training.

An Aeromagnetic Compensation Algorithm Based on a Deep Autoencoder

Magnetic compensation is a necessary step in the aeromagnetic data processing. While the aeromagnetic compensation model is a linear regression model, the multicollinearity of the variables in the model reduces the accuracy of the compensation model. To solve this problem, we propose a deep autoencoder (DAE) aeromagnetic compensation algorithm. The DAE searches the direction of maximum change in the data by using the gradient descent backpropagation algorithm. The special structure of the encoder can compress the representation of the coefficient matrix and extract data features, thereby weakening the correlation between the coefficient matrix variables. The features obtained after dimension reduction are used in the compensation calculation. We validate the DAE algorithm by applying it to data collected by an unmanned aerial vehicle and demonstrate that the DAE magnetic compensation results are better than those of the principal component analysis algorithm.

Recursive multilayer perceptron-based data-driven identification for a parameterized polarization model of rechargeable Li-ion battery

A battery model is intended to reflect the dynamics of a battery in terms of terminal voltage under different stress factors. The terminal voltage further depends on the open-circuit voltage (OCV) and polarizations of the battery. Therefore, an important concern of battery modeling is how to accurately map the impact of stress factors on the OCV and polarizations of the battery. Commonly, the OCV curve obtained experimentally by the DC pulse technique is fitted empirically to model the OCV as a function of the operating conditions by using different regression techniques, which may be suitable for offline modeling. However, for online implementation, intelligent fitting techniques are necessary to adjust the model according to changing conditions. Similarly, different combinations of resistors and capacitors are selected to model the polarizations, and these components are parameterized by curve fitting of the terminal voltage. Although the cumulative effect of the components may result in good curve fitting, the first-principle justification of the selection of components is lacking. To address this problem, data-driven identification of the parametric model of a battery and its subsequent parameterization by a multilayer perceptron (MLP) with a gradient-descent back-propagation algorithm are introduced. The proposed algorithm is intelligent, and it provides justification for the selection of components to model polarizations. Furthermore, this algorithm can be easily extended to develop a model adaptable to changes in battery life and operating conditions.

A Classification Framework using a Diverse Intensified Strawberry Optimized Neural Network (DISON) for Clinical Decision-making

A novel classification framework for clinical decision making that uses an Extremely Randomized Tree (ERT) based feature selection and a Diverse Intensified Strawberry Optimized Neural network (DISON) is proposed. DISON is a Feed Forward Artificial Neural Network where the optimization of weights and bias is done using a two phase training strategy. Two algorithms namely Strawberry Plant Optimization (SPO) algorithm and Gradient-descent Back-propagation algorithm are used sequentially to identify the optimum weights and bias. The novel two phase training method and the stochastic duplicate-elimination strategy of SPO helps in addressing the issue of local optima associated with conventional neural networks. The relevant attributes are selected based on the feature importance values computed using an ERT classifier.Vertebral Column, Pima Indian diabetes (PID), Cleveland Heart disease (CHD) and Statlog Heart disease (SHD) datasets from the University of California Irvine machine learning repository are used for experimentation. The framework has achieved an accuracy of 87.17% for Vertebral Column, 90.92% for PID, 93.67% for CHD and 94.5% for SHD. The classifier performance has been compared with existing works and is found to be competitive in terms of accuracy, sensitivity and specificity. Wilcoxon test confirms the statistical superiority of the proposed method.

İKİ BOYUTLU FONKSİYONEL KADEMELENDİRİLMİŞ PLAKALARIN YAPAY SİNİR AĞI ÖĞRENME ALGORİTMALARI İLE ISIL GERİLME MODELLEMESİ

Belirli bir hacimsel dağılım fonksiyonuna göre oluşturulan fonksiyonel kademelendirilmiş malzemeler (FKM) günümüz yüksek sıcaklık uygulamalarına dayanımlı malzeme üretiminde önemli bir yere sahiptir. Amaç fonksiyonuna bağlı olarak FKM’de maksimum çalışma performansı, yapısal varyasyonlar ve emniyetli gerilme değerleri gibi önemli özelliklerin sağlanabilmesi için hacimsel dağılımın belirlenmesi çok önemlidir. Hacimsel dağılımın belirlenmesi ve uygun hacimsel dağılımın test edilebilmesi için nümerik analiz yöntemleri kullanılmaktadır. Bu çalışmada, sonlu farklar metodu ile iki boyutlu fonksiyonel kademelendirilmiş plakaların ısı akısı tesirinde ısıl-mekanik davranışının tespit edilmesinde önemli bir parametre olan eşdeğer gerilme seviyeleri için malzemenin hacimsel dağılımına bağlı modeller oluşturulmuştur. Bu modeller yapay sinir ağında iki farklı eğitim algoritması ile elde edilmiştir. Bu eğitim algoritmaları Levenberg-Marquart ve Geriye Yayılım algoritmasıdır. FKM’lerin hacimsel dağılımını belirleyen kompozisyonel gradyant üst değerleri n ve m’dir. Sonlu farklar metodu ile sayısal analizde n ve m değerlerini belirlemek 1800 s iken Yapay Sinir ağı modeli ile 900 s olup verimlilik önemli derecede artmaktadır. Kullanılan eğitim algoritmalarının iş-zaman-performans değerleri açısından ileride yapılacak bilimsel çalışmalar için fikir verici nitelikte olması önemlidir. Önerilen eğitimli modeller henüz seri üretimi gerçekleştirilemeyen FKM için hem üretimde hem de yapılacak teorik çalışmalarda optimum hacimsel dağılıma ulaşmada yol gösterici olacaktır.

Image Features Based Intelligent Apple Disease Prediction System

A pattern classifier (PC) is used to solve a variety of non-separable and complex computing problems. One of the key problems is to efficiently predict a type of disease in a typical fruit tree. The timely and accurately predicted disease in an apple tree may help a farmer to take appropriate preventive measures in advance. In this article, an apple disease diagnosis system is developed to predict the apple scab and leaf/spot blight diseases. In this article, low level and shape-based features are used for the development of an intelligent apple disease prediction system. First, the key image features like entropy, energy, inverse difference moment (IDM), mean, standard deviation (SD), perimeter, etc., are extracted from the apple leaf images. The model for the proposed system is trained by using multi-layer perceptron (MLP) pattern classifier and eleven apple leaves image features. The Gradient descent back-propagation algorithm is used for building the intelligent system to carry out the pattern classification. The proposed system is tested using some random samples and exhibits excellent diagnosis accuracy of 99.1%. The sensitivity of the proposed prediction model is 98.1% and specificity of ~99.9%.

Optimization of Interval Type-2 Fuzzy Logic System for Software Reliability Prediction

Since real world application is fraught with high amount of uncertainty, such as applicable to software reliability, there should be a method of handling the uncertainty. This paper presents a model to properly handle uncertainty in software data for effective prediction of the reliability of software at the early phase of software development process. In this paper we employed a Takagi-Sugeno-Kang (TSK)-based interval type 2 fuzzy logic systems with artificial neural network learning for the prediction of software reliability. The degree of membership grades of the interval type 2 fuzzy sets (IT2FSs) are obtained using interval type Gaussian membership function with fixed mean and uncertain standard deviation. The parameters of the IT2FLS membership functions are optimized using gradient descent (GD) back-propagation algorithm. As inputs to the system, reliability relevant software requirement metrics and the software size metrics are used. The proposed new approach makes use of qualitative data of requirement metrics of twenty three real software projects to examine its predictive ability. The performance of the model is evaluated using five performance metrics and found to provide better results when compared with existing approaches.

A Robust Online Adaptive B-Spline MPPT Control of Three-Phase Grid-Coupled Photovoltaic Systems Under Real Partial Shading Condition

This work contributes to the research on three-phase photovoltaic systems via a new more robust online adaptive neuro-fuzzy maximum power point tracking (MPPT) control technique considering real partial shading and load conditions. The trapping in local minima and high computational cost in the existing neuro-fuzzy structure are addressed by incorporating B-spline function in the proposed control method. The system parameters are adjusted online via an adaptive neuro-fuzzy inference system rules acquired from the MPPT error. The optimization part of the proposed control law is performed through an online learning gradient-descent back-propagation algorithm. The superiority of the proposed control in terms of energy conversion efficiency, MPPT error, and output power is checked under the same operating conditions with well-known used traditional and intelligent MPPT control algorithms. Finally, the robustness of the proposed control is confirmed through a complete day simulation and comparison indexes.

Fuzzy model predictive control of normalized air-to-fuel ratio in internal combustion engines

In this paper, a fuzzy model predictive controller is developed to reduce the emission pollutants in spark ignition internal combustion engines. The path to this control goal is regulating the amount of normalized air-to-fuel ratio in the engine. In order to generate the simulation data, mean value engine model is simulated. To approximate the nonlinear and fast time-varying dynamics of the engine, a modified fuzzy relational model is trained offline in batch mode. For training, gradient descent back propagation algorithm along with evolutionary asexual reproduction optimization algorithm is used. Nonlinear structure of the fuzzy model of the engine imposes nonlinear optimization to produce control signals. Hence, gradient descent algorithm is used to generate online control signals. The effectiveness and robustness of the controller are evaluated through simulations.

Application of Artificial Intelligence Approach in Modeling Surface Quality of Aerospace Alloys in WEDM Process

In this work, artificial neural network (ANN) model is developed for prediction of surface roughness (SR) in wire-cut electrical discharge machining (WEDM) of Inconel 825 aerospace alloy. Four process parameters viz., pulse on time (Ton), pulse off time (Toff), peak current (Ip) and servo voltage (SV) are investigated using Box Behnken experimental design.Parametric variation shows that improved SR can be obtained at low levels of Ton and SV. A multi-layer feed forward ANN architecture 4-16-1 working on gradient descent back propagation algorithm isfound optimum and is statistically validated by conducting hypothesis tests. The developed model predicted with average 6.38% error and model accuracy is recorded as 93.62%. ANOVA showed that Ton is the most significant factor affecting SR with 76.12% contribution; followed by SV and Toff respectively with 7.18% and 5.3% contributions. The predictive capability of the developed ANN model is found encouraging and can be effectively used for predicting SR in WEDM of Inconel 825.

Ultrasonic sensing of pistachio canopy for low-volume precision spraying

Effective volume rate of pesticide application on a site-specific basis can reduce the amount of agrochemicals used in precision horticulture. The prototype sprayer used in this study provided volume application rate adapted to the canopy volume in pistachio orchards on a real-time and continuous basis. An electronic control system for the detection and estimation of tree canopy dimensions was designed for application rate adjustment. Three ultrasonic ranging USS3 sensors were utilized to estimate the distance to the target at three different heights. A MLP neural network with gradient-descent back-propagation algorithm, tangent-sigmoid transfer function, and 3-7-6 topology was used for volume estimation of tree sections. Training and validation errors as well as R2 values indicated the reliability of the network for volume prediction. Results of T-test for comparing the number of spray droplet impacts, coverage of (artificial) target, spray quality parameter and relative span factor between variable-rate and conventional spraying were not significant which indicates the consistency of spray distribution in selective application. Experiments showed a reduction in pesticide usage of about 34.5% by means of variable-rate technology (41.3, 25.6 and 36.5, respectively for the top, middle, and bottom sections of tree canopy). Precise application of agrochemicals reduces both costs and environmental pollution by supporting a decrease in the amount of delivered spray.

Use of Artificial Neural Network for Industrial Prediction of Final Temperature of Chicken Carcasses

AbstractIn this work, an artificial neural network (ANN) was used to predict the final temperature of chicken carcasses in an industrial scale. For this purpose, temperatures of chicken carcasses were obtained in a slaughterhouse at the end of the cooling process. The variables considered to influence the temperature of carcasses were average carcass weight, prechiller and chillers velocities, prechiller and chillers average temperatures, the absence or presence of bubbles in prechiller and chiller and bubbles intensity, which were considered as input for the ANN. For training and validation of the feed forward neural network, the above inputs were combined to minimize the weighted sum of the squares of the residues, being tested 10 transfer functions, three training algorithms and two different architectures. The best result was obtained using an ANN composed of two hidden layer (10 nodes in each layer), radial bias as transfer function and gradient descent backpropagation algorithm for training. Using this architecture, the regression coefficient was 0.9265. Even with many variables affecting the industrial cooling process of the chicken carcasses, the ANN developed showed satisfactory fitting of the final temperature of chicken carcasses. This model can be further used for optimization purposes, improving the process quality of chicken carcasses as well as process profitability.Practical ApplicationThe model used in this study can be further used for control and optimization purposes, improving the process quality of chicken carcasses as well as process profitability.

Robust control of wheel slip in anti-lock brake system of automobiles

In this paper, performances of two model-free control systems including Fuzzy Logic Control (FLC) and Neural Predictive Control (NPC) on tracking performance of wheel-slip in Anti-lock Braking System (ABS) are compared. As an accurate and control oriented model, a half vehicle model is developed to generate extensive simulation data of the braking system. Brake system identification is preformed through a Perceptron neural networks model of brake system which is trained with offline data by Gradient Descent Back Propagation (GDBP) algorithm. In order to reduce the time cost of the calculations and improving the robustness of closed loop control system, an online Perceptron neural network adaptively generates the optimum control actions. By a comparative simulation analysis it is shown that the NPC system has a better tracking performance, shorter stopping time and distance than the FLC controllers. The robustness of the proposed control systems are evaluated under ±25 % uncertainty. It is shown that the NPC system is more robust against both exogenous disturbances and modeling uncertainties than the FLC system.

Fault-tolerant nonlinear adaptive flight control using sliding mode online learning

An expanded nonlinear model inversion flight control strategy using sliding mode online learning for neural networks is presented. The proposed control strategy is implemented for a small unmanned aircraft system (UAS). This class of aircraft is very susceptible towards nonlinearities like atmospheric turbulence, model uncertainties and of course system failures. Therefore, these systems mark a sensible testbed to evaluate fault-tolerant, adaptive flight control strategies. Within this work the concept of feedback linearization is combined with feed forward neural networks to compensate for inversion errors and other nonlinear effects. Backpropagation-based adaption laws of the network weights are used for online training. Within these adaption laws the standard gradient descent backpropagation algorithm is augmented with the concept of sliding mode control (SMC). Implemented as a learning algorithm, this nonlinear control strategy treats the neural network as a controlled system and allows a stable, dynamic calculation of the learning rates. While considering the system's stability, this robust online learning method therefore offers a higher speed of convergence, especially in the presence of external disturbances. The SMC-based flight controller is tested and compared with the standard gradient descent backpropagation algorithm in the presence of system failures.

STUDYING THE EFFECT OF ADAPTIVE MOMENTUM IN IMPROVING THE ACCURACY OF GRADIENT DESCENT BACK PROPAGATION ALGORITHM ON CLASSIFICATION PROBLEMS

Despite being widely used in the practical problems around the world, Gradient Descent Back-propagation algorithm comes with problems like slow convergence and convergence to local minima. Previous researchers have suggested certain modifications to improve the convergence in gradient Descent Back-propagation algorithm such as careful selection of input weights and biases, learning rate, momentum, network topology, activation function and value for 'gain' in the activation function. This research proposed an algorithm for improving the working performance of back-propagation algorithm which is 'Gradient Descent with Adaptive Momentum (GDAM)' by keeping the gain value fixed during all network trials. The performance of GDAM is compared with 'Gradient Descent with fixed Momentum (GDM)' and 'Gradient Descent Method with Adaptive Gain (GDM-AG)'. The learning rate is fixed to 0.4 and maximum epochs are set to 3000 while sigmoid activation function is used for the experimentation. The results show that GDAM is a better approach than previous methods with an accuracy ratio of 1.0 for classification problems like Wine Quality, Mushroom and Thyroid disease.

The Effect of Adaptive Gain and Adaptive Momentum in Improving Training Time of Gradient Descent Back Propagation Algorithm on Classification Problems

The back propagation algorithm has been successfully applied to wide range of practical problems. Since this algorithm uses a gradient descent method, it has some limitations which are slow learning convergence velocity and easy convergence to local minima. The convergence behaviour of the back propagation algorithm depends on the choice of initial weights and biases, network topology, learning rate, momentum, activation function and value for the gain in the activation function. Previous researchers demonstrated that in ‘feed forward’ algorithm, the slope of the activation function is directly influenced by a parameter referred to as ‘gain’. This research proposed an algorithm for improving the performance of the current working back propagation algorithm which is Gradien Descent Method with Adaptive Gain by changing the momentum coefficient adaptively for each node. The influence of the adaptive momentum together with adaptive gain on the learning ability of a neural network is analysed. Multilayer feed forward neural networks have been assessed. Physical interpretation of the relationship between the momentum value, the learning rate and weight values is given. The efficiency of the proposed algorithm is compared with conventional Gradient Descent Method and current Gradient Descent Method with Adaptive Gain was verified by means of simulation on three benchmark problems. In learning the patterns, the simulations result demonstrate that the proposed algorithm converged faster on Wisconsin breast cancer with an improvement ratio of nearly 1.8, 6.6 on Mushroom problem and 36% better on Soybean data sets. The results clearly show that the proposed algorithm significantly improves the learning speed of the current gradient descent back-propagatin algorithm.

On-line Network Reconfiguration for Enhancement of Voltage Stability in Distribution Systems Using Artificial Neural Networks

Network reconfiguration for maximizing voltage stability is the determination of switching-options that maximize voltage stability the most for a particular set of loads on the distribution systems, and is performed by altering the topological structure of distribution feeders. Network reconfiguration for time-varying loads is a complex and extremely nonlinear optimization problem which can be effectively solved by Artificial Neural Networks (ANNs), as ANNs are capable of learning a tremendous variety of pattern mapping relationships without having a priori knowledge of a mathematical function. In this paper a generalized ANN model is proposed for on-line enhancement of voltage stability under varying load conditions. The training sets for the ANN are carefully selected to cover the entire range of input space. For the ANN model, the training data are generated from the Daily Load Curves (DLCs). A 16-bus test system is considered to demonstrate the performance of the developed ANN model. The proposed ANN is trained using Conjugate Gradient Descent Back-propagation Algorithm and tested by applying arbitrary input data generated from DLCs. The test results of the ANN model are found to be the same as that obtained by off-line simulation. The enhancement of voltage stability can be achieved by the proposed method without any additional cost involved for installation of capacitors, tap-changing transformers, and the related switching equipment in the distribution systems. The developed ANN model can be implemented in hardware using the neural chips currently available.

Training Feedforward Neural Networks: An Algorithm Giving Improved Generalization

An algorithm is derived for supervised training in multilayer feedforward neural networks. Relative to the gradient descent backpropagation algorithm it appears to give both faster convergence and improved generalization, whilst preserving the system of backpropagating errors through the network. Copyright © 1996 Elsevier Science Ltd.