Feed-forward Neural Network Algorithm Research Articles

Using two feed-forward layers for fault detection of hole

Automated assembly technology is a major component of modern aerospace manufacturing. Hole-Making Robot (HMR)s can significantly improve the efficiency of the aircraft manufacturing and can also ensure reliability of processing. The machining accuracy of the HMR depends on the precision of the mechanism, the accuracy of hole-positioning, and the vertical degree of the hole. Evaluating the precision of the hole system requires a complete set of inspection systems to measure the relevant hole parameters. Visual detection technology can provide more information on the object, which in theory is better suited to simulate real world applications. Due to the rapid development of current visual technology and the actual demand of aviation holes, this paper attempts to apply Convolution Neural Network (CNN) technology to the concrete task of visual inspection and improve the performance of visual inspection systems. Hole classification and hole flaw detection are realized via two feed-forward layers. We created a dataset of 1300 images of holes for testing. These were collected from various sources. The highest accuracy was 98.15% and was achieved using the two feed-forward neural network algorithm.

Constitutive models of magnetorheological fluids having temperature-dependent prediction parameter

This work presents constitutive models of magnetorheological (MR) fluids, which can predict the shear and dynamic yield stress depending on temperature. Two existing models, the Herschel–Bulkley rheological and power law model, which are frequently used in MR fluid research, are adopted and modified to take the temperature into account. A new constitutive model of MR fluids is developed using the extreme learning machine (ELM) method. In this development, among many machine learning approaches, a simple and efficient learning algorithm for a single hidden layer feed-forward neural network (SLFN) is adopted and applied to the rheological model of MR fluids. The temperature, shear rate, and magnetic field are treated as inputs, and the shear stress is taken as an output. After formulating the models associated with experimental coefficients, the two most important properties of MR fluids; the shear and yield stress are predicted and compared with the measured values. The prediction accuracy for the field-dependent rheological properties of MR fluids in several different temperatures is evaluated and compared. It is shown that the ELM model developed in this work provides the best accuracy, followed by two other modified constitutive equations.

Dynamic group optimisation algorithm for training feed-forward neural networks

Feed-forward neural networks are efficient at solving various types of problems. However, finding efficient training algorithms for feed-forward neural networks is challenging. The dynamic group op ...

Radar HRRP Target Recognition Based on Stacked Autoencoder and Extreme Learning Machine.

A novel radar high-resolution range profile (HRRP) target recognition method based on a stacked autoencoder (SAE) and extreme learning machine (ELM) is presented in this paper. As a key component of deep structure, the SAE does not only learn features by making use of data, it also obtains feature expressions at different levels of data. However, with the deep structure, it is hard to achieve good generalization performance with a fast learning speed. ELM, as a new learning algorithm for single hidden layer feedforward neural networks (SLFNs), has attracted great interest from various fields for its fast learning speed and good generalization performance. However, ELM needs more hidden nodes than conventional tuning-based learning algorithms due to the random set of input weights and hidden biases. In addition, the existing ELM methods cannot utilize the class information of targets well. To solve this problem, a regularized ELM method based on the class information of the target is proposed. In this paper, SAE and the regularized ELM are combined to make full use of their advantages and make up for each of their shortcomings. The effectiveness of the proposed method is demonstrated by experiments with measured radar HRRP data. The experimental results show that the proposed method can achieve good performance in the two aspects of real-time and accuracy, especially when only a few training samples are available.

Informatics Solution for Energy Efficiency Improvement and Consumption Management of Householders

Although in 2012 the European Union (EU) has promoted energy efficiency in order to ensure a gradual 20% reduction of energy consumption by 2020, its targets related to energy efficiency have increased and extended to new time horizons. Therefore, in 2016, a new proposal for 2030 of energy efficiency target of 30% has been agreed. However, during the last years, even if the electricity consumption by households decreased in the EU-28, the largest expansion was recorded in Romania. Taking into account that the projected consumption peak is increasing and energy consumption management for residential activities is an important measure for energy efficiency improvement since its ratio from total consumption can be around 25–30%, in this paper, we propose an informatics solution that assists both electricity suppliers/grid operators and consumers. It includes three models for electricity consumption optimization, profiles, clustering and forecast. By this solution, the daily operation of appliances can be optimized and scheduled to minimize the consumption peak and reduce the stress on the grid. For optimization purpose, we propose three algorithms for shifting the operation of the programmable appliances from peak to off-peak hours. This approach enables the supplier to apply attractive time-of-use tariffs due to the fact that by flattening the consumption peak, it becomes more predictable, and thus improves the strategies on the electricity markets. According to the results of the optimization process, we compare the proposed algorithms emphasizing the benefits. For building consumption profiles, we develop a clustering algorithm based on self-organizing maps. By running the algorithm for three scenarios, well-delimited profiles are obtained. As for the consumption forecast, highly accurate feedforward artificial neural networks algorithm with backpropagation is implemented. Finally, we test these algorithms using several datasets showing their performance and integrate them into a web-service informatics solution as a prototype.

Back Propagation Neural Network (BPNN) Based Recognition Of Handwritten Mathematical Equations

The recognition of handwritten mathematical symbols and equations are the critical and challenging issue in the field of pattern recognition. It is the need to recognize complicated handwritten mathematical equations viz. law of gravity, convolution integral etc. The issues like overwriting of symbols, characters etc. are identified and solved it by selecting best classifier to improve recognition rate. The machine learning approach with enhanced multi layer precentor feed forward back propagation neural network algorithm with an offline mode of recognition has been used to improve throughput, accuracy and overall efficiency of mathematical equations recognition. The hybrid features extracted viz. centroid, boundary box, zoning density, line segment etc. and gradient descent with momentum training algorithm has been used. Adaptive learning is used to carry out the experiment on numerous kinds of equations. Through experimental result, the system is evaluated and illustrated which shows the significant improvement 93.5 % accuracy in recognition of simple as well as complicated mathematical equations. In future current methodology will be the key factor to initiate paperless work and digital world.

A Modified Residual Extreme Learning Machine Algorithm and Its Application

Extreme learning machine (ELM) is a fast learning algorithm for the single-hidden layer feedforward neural networks. However, usually we cannot guarantee the stability of the ELM because the parameters of the ELM are generated randomly, such as its biases of the hidden layer and the connecting weights between the input layer and the hidden layer. Besides, it is hard for a single model to achieve high predicted accuracy on the dataset with low-quality data. In this paper, we first propose a modified residual ELM (R-ELM) to improve the ELM’s learning performance. In R-ELM, the first ELM is trained by the original dataset and the $m$ -th ( $m > 1$ ) ELM will be trained by the residuals between the ground truths and the predicted results of the previous ensemble model (with $m-1$ ELMs). R-ELM (with $m$ ELMs) is built in the direction of the error reduction by calculating the $m$ -th ELM’s optimal weight which is determined by the loss function of the R-ELM. As a result, R-ELM can remember almost all information of the training set. However, this ability does not assure a similar performance on the testing dataset. In view of this problem, we add ${L_{2}}$ regularization to the loss function of the R-ELM (RR-ELM) to avoid the overfitting problem of R-ELM. In RR-ELM, ${L_{2}}$ regularization is employed to encourage each ELM to ignore the unnecessary information of the training set. In order to verify the effectiveness of the two proposed algorithms, the real data from the blast furnace are engaged to perform the experiments. Experimental results illustrate that the proposed RR-ELM and the R-ELM are more stable than the single ELM. These results also demonstrate that the two proposed methods are more accurate than the average outputs of a group of ELMs, the ELM, and the support vector regression.

Automated Classification Method for Early Diagnosis of Alopecia Using Machine Learning

Artificial neural networks are being used in many fields for different purposes. Medical diagnosis is one of the major purposes. In the field of medical, classification plays an important role as the main aim of the doctor is to classify whether a person is suffering from the disease or not. The objective of this paper is to evaluate neural network for the detection of alopecia in human beings and to find the accuracy. With the help of proposed model, the clinical experts will be able to get a second opinion that will help them to take proper decisions for diagnosing the presence of this disease in patients. This second opinion is crucial due to many factors while doing disease identification. These factors include increased population, environmental pollution, growing demands for proper medication, and less availability of medical experts to cope up with this increasing demand. Also, the dynamic nature of disease symptoms plays an important role in correct diagnosis of a certain disease. The proposed system uses a feedforward artificial neural network and backpropagation algorithm to classify patients with alopecia and without alopecia. The evaluation of the proposed system is done with the help of performance plot as well as regression plot. Experimental results show that the accuracy of proposed system is 91% which is reliable enough for a clinical expert to make his decisions.

Optimal model for path loss predictions using feed-forward neural networks

In this paper, an optimal model is developed for path loss predictions using the Feed-Forward Neural Network (FFNN) algorithm. Drive test measurements were carried out in Canaanland Ota, Nigeria and Ilorin, Nigeria to obtain path loss data at varying distances from 11 different 1,800 MHz base station transmitters. Single-layered FFNNs were trained with normalized terrain profile data (longitude, latitude, elevation, altitude, clutter height) and normalized distances to produce the corresponding path loss values based on the Levenberg–Marquardt algorithm. The number of neurons in the hidden layer was varied (1–50) to determine the Artificial Neural Network (ANN) model with the best prediction accuracy. The performance of the ANN models was evaluated based on different metrics: Mean Absolute error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), standard deviation, and regression coefficient (R). Results of the machine learning processes show that the FNN architecture adopting a tangent activation function and 48 hidden neurons produced the least prediction error, with MAE, MSE, RMSE, standard deviation, and R values of 4.21 dB, 30.99 dB, 5.56 dB, 5.56 dB, and 0.89, respectively. Regarding generalization ability, the predictions of the optimal ANN model yielded MAE, MSE, RMSE, standard deviation, and R values of 4.74 dB, 39.38 dB, 6.27 dB, 6.27 dB, and 0.86, respectively, when tested with new data not previously included in the training process. Compared to the Hata, COST 231, ECC-33, and Egli models, the developed ANN model performed better in terms of prediction accuracy and generalization ability.

Prediction capabilities of mathematical models in producing a renewable fuel from waste cooking oil for sustainable energy and clean environment

The present work describes the comparison of biodiesel yield prediction by Response Surface Methodology (RSM) and Artificial Neural Network (ANN). The prediction models were developed based on three-level design of experiments conducted with waste cooking oil transesterified by varying four process parameters such as catalyst concentration, molar ratio, reaction time, and stirrer speed. The optimum reaction conditions were found to be 0.75% wt/wt catalyst concentration, 9:1 M ratio, 60 min reaction time and 500 rpm stirrer speed. For these optimum conditions, experimental fatty acid methyl ester (FAME) content of 95.05 ± 0.26% was obtained, which was in good agreement with the predicted yield. The RSM model was developed using Box-Behnken design and the ANN predictive model was developed using a feed-forward backpropagation neural network algorithm with 14 neurons in the hidden layer. The mathematical models of RSM and developed ANN were compared for biodiesel yield. The higher value of correlation coefficient (R2 = 0.99) and lower value of root mean square error (RMSE = 1.97) for ANN compared to RSM (R2 = 0.95 and RMSE = 2.71) evidently proved that ANN model is far better in predicting FAME content compared to the RSM model.

Multi-criteria decision making based architecture selection for single-hidden layer feedforward neural networks

Architecture selection is a fundamental problem in artificial neural networks, which could be treated as a decision making process that evaluates, ranks, and makes choices from a set of network structures. Traditional methods evaluate a network structure by designing a criterion based on a validation model or an error bound model. On one hand, the time complexity of a validation model is usually high; on the other hand, different validation models or error bound models may lead to different (even conflicting) results, which post challenges to the traditional single criterion-based architecture selection methods. In the area of decision making, many problems employed multiple criteria since the performance is better than using a single criterion. In this paper, we propose a multi-criteria decision making based architecture selection algorithm for single-hidden layer feedforward neural networks trained by extreme learning machine. Two criteria are incorporated into the selection process, i.e., training accuracy and the Q-value estimated by the localized generalization error model. The training accuracy reflects the capability of the model on correctly categorizing the known samples, and the Q-value estimated by localized generalization error model reflects the size of the neighbourhood of training samples in which the model can predict unseen samples with confidence. By achieving a trade-off between these two criteria, a new architecture selection algorithm is proposed. Experimental comparisons demonstrate the feasibility and effectiveness of the proposed method.

A Simple, Fast and Highly-Accurate Algorithm to Recover 3D Shape from 2D Landmarks on a Single Image.

Three-dimensional shape reconstruction of 2D landmark points on a single image is a hallmark of human vision, but is a task that has been proven difficult for computer vision algorithms. We define a feed-forward deep neural network algorithm that can reconstruct 3D shapes from 2D landmark points almost perfectly (i.e., with extremely small reconstruction errors), even when these 2D landmarks are from a single image. Our experimental results show an improvement of up to two-fold over state-of-the-art computer vision algorithms; 3D shape reconstruction error (measured as the Procrustes distance between the reconstructed shape and the ground-truth) of human faces is , cars is .0022, human bodies is .022, and highly-deformable flags is .0004. Our algorithm was also a top performer at the 2016 3D Face Alignment in the Wild Challenge competition (done in conjunction with the European Conference on Computer Vision, ECCV) that required the reconstruction of 3D face shape from a single image. The derived algorithm can be trained in a couple hours and testing runs at more than 1,000 frames/s on an i7 desktop. We also present an innovative data augmentation approach that allows us to train the system efficiently with small number of samples. And the system is robust to noise (e.g., imprecise landmark points) and missing data (e.g., occluded or undetected landmark points).

Fast convergent distributed cooperative learning algorithms over networks

Abstract This paper aims to design a fast convergent distributed cooperative learning (DCL) algorithm for feedforward neural networks with random weights (FNNRWs) over undirected and connected networks. First, a continuous-time fast convergent DCL algorithm is proposed, whose finite-time convergence is guaranteed based on the Lyapunov method. Second, we extend this algorithm to a discrete-time form by using the fourth-order Runge–Kutta method. Compared with the distributed alternating direction method of multipliers (ADMM) and the Zero-Gradient-Sum-based (ZGS-based) algorithms, the proposed algorithm has high learning capability and convergence speed. Simulation results demonstrate that the proposed algorithm has fast convergence rate, and the convergence rate may be adjusted by properly selecting some tuning parameters.

Ensemble dropout extreme learning machine via fuzzy integral for data classification

Abstract Extreme learning machine (ELM) is a simple but efficient algorithm for training single hidden layer feed-forward neural networks (SLFNs) with fast speed and good generalization ability. ELM has been successfully applied to many fields, such as pattern recognition, computer vision, biological information processing, etc. However, there are two problems in ELM. the first one is architecture selection, the second one is prediction instability. In order to deal with the two problems, based on dropout technique, an ensemble learning method is proposed in this paper. The proposed method can solve the first problem and can improve prediction stability. Our experimental results and statistical analysis on 14 data sets confirm this conclusion. Furthermore, our experimental results also show that the proposed approach outperforms the original ELM on prediction stability and classification accuracy.

LANN-SVD: A Non-Iterative SVD-Based Learning Algorithm for One-Layer Neural Networks.

In the scope of data analytics, the volume of a data set can be defined as a product of instance size and dimensionality of the data. In many real problems, data sets are mainly large only on one of these aspects. Machine learning methods proposed in the literature are able to efficiently learn in only one of these two situations, when the number of variables is much greater than instances or vice versa. However, there is no proposal allowing to efficiently handle either circumstances in a large-scale scenario. In this brief, we present an approach to integrally address both situations, large dimensionality or large instance size, by using a singular value decomposition (SVD) within a learning algorithm for one-layer feedforward neural network. As a result, a noniterative solution is obtained, where the weights can be calculated in a closed-form manner, thereby avoiding low convergence rate and also hyperparameter tuning. The proposed learning method, LANN-SVD in short, presents a good computational efficiency for large-scale data analytic. Comprehensive comparisons were conducted to assess LANN-SVD against other state-of-the-art algorithms. The results of this brief exhibited the superior efficiency of the proposed method in any circumstance.

A Hybrid Simulated Annealing and Back-Propagation Algorithm for Feed forward Neural Network to Detect Credit Card Fraud

Due to the ascent and fast development of E-commerce, utilization of credit cards for online buys has significantly expanded, and it brought about a blast in the credit card fraud. As credit card turns into the most prevalent method of installment for both online and also normal buy, cases of fraud associated with it are additionally rising. In actuality, false exchanges are scattered with veritable exchanges, and basic example for coordinating procedures is not frequently adequate to identify those frauds accurately. Usage of effective fraud recognition frameworks has in this manner gotten to be basic for all credit card distributing banks to decrease their losses. Many current systems based on artificial intelligence, Fuzzy logic, machine learning, data mining, sequence alignment, genetic programming, and so on have advanced in distinguishing different credit card fake transactions. A reasonable seeing on all these methodologies will absolutely lead to an efficient credit card fraud detection framework. This paper suggested an anomaly detection model based on a hybrid simulated annealing (SA) and back-propagation algorithm for feed-forward neural network (FFNN), which joined the significant global searching capability of SA with the precise local searching element of back-propagation FFNNs to improve theinitial weights of a neural network toward getting a better result for detection fraud. Index Terms: Artificial Neural Network, Back-propagation, Back-propagation Feed-forward Neural Network, Feed-forward Neural Network, Simulated Annealing, Simulated Annealing-back-propagation Feed-forward Neural Network

A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

In this paper, a new learning algorithm is proposed for complex-valued feedforward neural networks (CVFNNs). The basic idea of this algorithm is that the descent directions of the cost function with respect to complex-valued parameters are calculated by limited-memory BFGS algorithm and the learning step is determined by Armijo line search method. Since the approximation of Hessian matrix is calculated by utilizing the information of the latest several iterations, the memory efficiency is improved. To keep away from the saturated ranges of activation functions, some gain parameters are adjusted together with weights and biases. Compared with some existing learning algorithms for CVFNNs, the convergence speed is faster and a deeper minima of the cost function can be reached by the developed algorithm. In addition, the effects of initial values of weights and biases on the efficiency and convergence speed of the learning algorithm are analyzed. The performance of the proposed algorithm is evaluated in comparison with some existing classifiers on a variety of benchmark classification problems. Experimental results show that better performance is achieved by our algorithm with relatively compact network structure.

Fault diagnosis for wind turbine based on improved extreme learning machine

A fault diagnosis method based on improved extreme learning machine (IELM) is proposed to solve the weakness (weak generalization ability, low diagnostic rate) of traditional fault diagnosis with feedforward neural network algorithm. This method fuses signal feature vectors, extracts six parameters as the principal component analysis (PCA) variables, and calculates correlation coefficient matrix among the variables. The weight values of control parameters in the extreme learning model are dynamically adjusted according to the test samples’ constantly changing. Consequently, the weight fixed drawback in the original model can be remedied. A fault simulation experiment platform for wind turbine drive system is built, eight kinds of fault modes are diagnosed by the improved extreme learning model, and the result is compared with that of other machine learning methods. The experiment indicates that the method can enhance the accuracy and generalization ability of diagnosis, and increase the computing speed. It is convenient for engineering application.

An Improved ACOBP Algorithm for Load Forecasting in Power System

In this paper, a new ant colony algorithm is introduced and a new feedforward network training strategy, ACOBP algorithm, is proposed for the first time, based on the traditional linear feedforward neural network and BP algorithm. In this paper, the network is used for load forecasting and the results of the example show that this method is feasible and reasonable. In the aspect of sample data processing, this paper has fully considered the influence of several main factors on the load variation, and quantified a series of weather factors, such as temperature and rainfall, and forecasted the important holiday separately. By analyzing the special law of load, the network parameters are adjusted and the precision of prediction is improved.

Two swarm intelligence approaches for tuning extreme learning machine

Extreme learning machine (ELM) is a new algorithm for training single-hidden layer feedforward neural networks which provides good performance as well as fast learning speed. ELM tends to produce good generalization performance with large number of hidden neurons as the input weights and hidden neurons biases are randomly initialized and remain unchanged during the learning process, and the output weights are analytically determined. In this paper, two swarm intelligence based metaheuristic techniques, viz. Artificial Bee Colony (ABC) and Invasive Weed Optimization (IWO) are proposed for tuning the input weights and hidden biases. The proposed approaches are called ABC-ELM and IWO-ELM in which the input weights and hidden biases are selected using ABC and IWO respectively and the output weights are computed using the Moore-Penrose (MP) generalized inverse. The proposed approaches are tested on different benchmark classification data sets and simulations show that the proposed approaches obtain good generalization performance in comparison to the other techniques available in the literature.