Multilayer Feed-forward Artificial Neural Network Research Articles

BP Artificial Neural Network and its Application Based on LM Algorithm

Artificial Neural Network (ANN) is a new type of intelligence system, which imitates the biological neural contracture. It is a never-unit-linked network based on the theory of biology nerve nets. It mainly researches the information distribution and signal process in the complicated multi-dimensional non-linear system, also the principle and method of signal processing, at the same time, applies them to solving the practical problems in engineering. The research on this subject combines the fundamental subjects such as biology, cognitive science, non-liner science with these engineering subjects like computer, electronics, AI, information processing, model recognizing and so on. It has a bright applying promise. BP neural network is a multi-layer feed-forward artificial neural network which is the most widely used and researched at present. It has simple structure, good non-linear quality, extremely high fitting accuracy, flexible and effective storage, and the hierarchy of model structure is easy to implement. In this paper, a simple and feasible BP neural network model, whose strong nonlinear ability is utilized, to predict the potential distribution of Mikania micrantha in Guangzhou. An improved model based on LM algorithm is proposed, and the results show that the potential distribution area and the actual distribution area have a good correspondence.

Prediction of the Tensile Response of Carbon Black Filled Rubber Blends by Artificial Neural Network.

The precise experimental estimation of mechanical properties of rubber blends can be a very costly and time-consuming process. The present work explores the possibilities of increasing its efficiency by using artificial neural networks to study the mechanical behavior of these widely used materials. A multilayer feed-forward back-propagation artificial neural network model, with a strain and the carbon black content as input parameters and stress as an output parameter, has been developed to predict the uniaxial tensile response of vulcanized natural rubber blends with different contents of carbon black in the form of engineering stress-strain curves. A novel procedure has been created for the simulation of the optimized artificial neural network model with input datasets generated by a regression model of an experimental dependence of tensile strain-at-break on the carbon black content in the investigated blends. Errors of the prediction of experimental stress-strain curves, as well as of tensile strain-at-break, tensile stress-at-break and M100 tensile modulus were estimated for all simulated stress-strain curves. The present study demonstrated that the performance of a developed neural network model to predict the stress-strain curves of rubber blends with different contents of carbon black is also exceptionally high in the case of a network that had never learned the input data, which makes it a suitable tool for extensive use in practice.

Prediction of the vertical force during FSW of AZ31 magnesium alloy sheets using an artificial neural network-based model

A multivariable empirical model based on an artificial neural network (ANN) was developed in order to predict the vertical force occurring during friction stir welding (FSW) of sheets in AZ31 magnesium alloy. To this purpose, FSW experiments were performed at different values of rotational and welding speeds, and the vertical force versus time curve was recorded during the different stages of the process by means of a dedicated sandwich dynamometer. Such results were used in the training stage of the artificial neural network-based model developed to predict vertical force versus time curves. A multi-layer feed forward ANN, using the back-propagation algorithm, consisting of the input layer with four input parameters (rotational speed, welding speed, rotational speed to welding speed ratio and processing time), two hidden layers with four neurons each, and the output layer with the vertical force as output, was built and trained. The generalization capability of the ANN was tested using a two-step procedure: in the former, the leave-one-out cross-validation method was used whilst, in the latter, curves not included in the training dataset were taken into account. The low values of the relative error and average absolute relative error, and the high correlation coefficients between predicted and experimental results have proven the excellent capability of the artificial neural network in modeling complex shape of the curve and in capturing the effect of the process parameters on the vertical force without a priori knowledge of the complex microstructural and mechanical mechanisms taking place during friction stir welding. Finally, the relationship between vertical force and processing time, at different welding and rotational speeds, was also predicted using the support vector machine algorithm and the results were compared with those given by the ANN-based model.

Energy modeling and saving potential analysis using a novel extreme learning fuzzy logic network: A case study of ethylene industry

Comprehensive energy modeling and saving potential analysis play a key role in sustainable development of complex petrochemical industry. However, it is difficult to make effective energy modeling and saving potential analysis due to the characteristics of uncertainty, high nonlinearity, and with noise of the data collected from the practical production. To deal with this problem, an energy modeling and saving potential analysis method using a novel extreme learning fuzzy logic network is proposed. In the proposed method, Mamdani type fuzzy inference system and multi-layer feedforward artificial neural network are integrated. First, the original ethylene production data is fused into a comprehensive energy consumption index. Then the index is fuzzified as outputs instead of precise values. Finally, an extreme learning algorithm based on Moore-Penrose Inverse is utilized to train the network efficiently. Three levels of energy efficiency of “low efficiency, median efficiency and high efficiency” can be effectively classified using the proposed method. For “low efficiency”, valid slack variables are predicted for finding the direction of improving energy efficiency and then analyzing the energy saving potential. The performance and the practicality of the proposed method are confirmed through an application of China ethylene industry. Simulation results show that low-efficiency samples can be effectively improved to be high-efficiency samples and the energy saving potential in terms of the crude oil reduction amount is indicted as 8.82%.

Classification of Arrhythmia using Wavelet Transform and Neural Network Model

Cardiovascular diseases are a major cause of death. Change in normal human heart beat may result in different types of cardiac arrhythmias. An Irreversible damage to the heart is possible. In this paper a method is proposed to classify different arrhythmias and normal sinus rhythm, through a combination of wavelet Transform and Artificial Neural Networks (ANN) accurately and efficiently. Adaptive filtering using Recursive Least squares (RLS) adaptive algorithm is utilized to nullify AC and DC noises from the sample ECG signal set. ECG data’s are collected from MITBIH database. As ECG signal is a non- stationary signal wavelet transform is used to decompose the signal at various resolutions. This allows accurate detection and extraction of features. In our approach, discrete wavelet transforms (DWT) coefficients set is obtained from wavelet decomposition which would contain the maximum information about the arrhythmia. RR interval, QRS duration, PR duration is extracted from the wavelet decomposition. With these parameters classification of arrhythmia is done. Multilayer feed forward ANNs employ error back propagation (EBP) learning algorithm were trained and tested using the extracted parameters are used for training and testing the error back propagation (EBP) algorithm. Multilayer feed forward ANNs are employed through this EBP learning algorithm. This classification is done for 84 patient samples. The overall accuracy of our approach is 98.8%.

Approximation of the process of changes in deformation of land surface using artificial neural networks

Artificial neural networks are an interesting method for modelling phenomena, including spatial phenomena, which are difficult to describe with known mathematical models. The properties of neural networks enable their practical application for solving such problems as: approximation, interpolation, identification and classification of patterns, compression, prediction, etc. The article presents the use of multilayer feedforward artificial neural networks for describing the process of changes in land surface deformation in the area of the Legnica-Głogów Copper Mining Centre, located in the southern part of the Fore Sudetic Monocline. Results provided by geodesic monitoring, which consists of land surveying and interpreting data obtained in this way, are undoubtedly significant in terms of identifying the impact of mining on the land surface the results of measurements carried out by precise levelling in the years 19672014 were used to determine changes in land deformation in the Legnica-Głogów Copper Mining Centre. The concept of a flexible reference system was used to assess the stability of points in the measurement and control network stabilized in order to determine vertical displacements. However, the reference system itself was identified on the basis of the critical value of the increment of the square of the norm of corrections to the observations.

Utilization of the LMS Algorithm to Filter the Predicted Course by Means of Neural Networks for Monitoring the Occupancy of Rooms in an Intelligent Administrative Building

For monitoring the occupancy of individual rooms in an intelligent administrative building (IAB), a wide variety of sensors can be used, by means of which the presence of a person in the monitored area can be determined. For a larger administrative-type building, installation of additional sensors means considerable investment costs. As the present-day standard, temperature or also humidity sensors are installed in individual IAB rooms. The paper describes the proposed method for the determination of occupancy of the monitored area by means of prediction of the course of CO2(ppm) from the measured values of humidity rH(%), indoor temperature Ti(°C) and outdoor temperature To(°C), using the gradient algorithm of back-propagation of error for adaptation of the multilayer feedforward Artificial Neural Network (ANN) in the IAB areas with utilization of the Bayesian regularization method (BRM) to obtain information on the occupancy of individual rooms. The LMS algorithm was used to filter the predicted course in order to determine the occupancy of the monitored areas more precisely. The advantage of the proposed method is the utilization of common operating sensors to obtain information on the state of operational-technical functions in the IAB for the purpose of optimum control of the operational-technical functions of the IAB on the basis of predictable needs of persons using the IAB areas.

Approximation of the process of changes in deformation of land surface using artificial neural networks

Artificial neural networks are an interesting method for modelling phenomena, including spatial phenomena, which are difficult to describe with known mathematical models. The properties of neural networks enable their practical application for solving such problems as: approximation, interpolation, identification and classification of patterns, compression, prediction, etc. The article presents the use of multilayer feedforward artificial neural networks for describing the process of changes in land surface deformation in the area of the Legnica-Głogów Copper Mining Centre, located in the southern part of the Fore Sudetic Monocline. Results provided by geodesic monitoring, which consists of land surveying and interpreting data obtained in this way, are undoubtedly significant in terms of identifying the impact of mining on the land surface the results of measurements carried out by precise levelling in the years 19672014 were used to determine changes in land deformation in the Legnica-Głogów Copper Mining Centre. The concept of a flexible reference system was used to assess the stability of points in the measurement and control network stabilized in order to determine vertical displacements. However, the reference system itself was identified on the basis of the critical value of the increment of the square of the norm of corrections to the observations.

Retrieval of fractional snow covered area from MODIS data by multivariate adaptive regression splines

In this paper, a novel approach to estimate fractional snow cover (FSC) from MODIS data in a complex and heterogeneous Alpine terrain is represented by using a state-of-the-art nonparametric spline regression method, namely, multivariate adaptive regression splines (MARS). For this purpose, twenty MODIS - Landsat 8 image pairs acquired between April 2013 and December 2016 over European Alps are used. Fifteen of the image pairs are employed during model training and five images are reserved as an independent test dataset. MARS models are trained by using MODIS top-of-atmosphere reflectance values of bands 1–7, normalized difference snow index, normalized difference vegetation index and land cover class as predictor variables. Reference FSC maps are generated from higher spatial resolution Landsat 8 binary snow cover maps. Multilayer feedforward artificial neural network (ANN) models are also trained by using the same input data. During the training and the testing, the effects of the training data size and the sampling type on the predictive performance of ANN and MARS models are investigated. An additional search is also conducted to reveal whether the choice of the transfer function used in the output layer of ANN has a significant contribution to the network's FSC mapping performance. The final ANN and MARS FSC products are at 500m spatial resolution. The results on the independent test scenes indicate that the developed ANN models with linear and hyperbolic tangent transfer functions in the output layer and the MARS models are in good agreement with reference FSC data with the same average values of R=0.93. In contrast, the standard MODIS snow fraction product, namely, MOD10 FSC, exhibits slightly poorer performance with average R=0.88. The proposed MARS approach is statistically proven to have the same performance with ANN, yet it is computationally more efficient in model building.

An Optimized Back Propagation Learning Algorithm with Adaptive Learning Rate

Back Propagation (BP) is commonly used algorithm that optimize the performance of network for training multilayer feed-forward artificial neural networks. However, BP is inherently slow in learning and it sometimes gets trapped at local minima. These problems occur mailnly due to a constant and non-optimum learning rate (a fixed step size) in which the fixed value of learning rate is set to an initial starting value before training patterns for an input layer and an output layer. This fixed learning rate often leads the BP network towrds failure during steepest descent. Therefore to overcome the limitations of BP, this paper introduces an improvement to back propagation gradient descent with adapative learning rate (BPGD-AL) by changing the values of learning rate locally during the learning process. The simulation results on selected benchmark datasets show that the adaptive learning rate significantly improves the learning efficiency of the Back Propagation Algorithm

Artificial Neural Network-Based Modeling for Impact Energy of Cast Duplex Stainless Steel

The exploitation of artificial neural network as a computational technique in predicting the impact energy of cast duplex stainless steels based on its chemical composition is reported in this research work. Two hundred and twenty melts of duplex stainless steel of different compositions were casted, heat-treated and tested for Charpy impact test. A multilayer feed forward ANN model was developed based on 75% of the available chemical compositions of duplex stainless steel as input and impact energy in joules as output. The prediction efficiency of the developed models was calculated based on mean absolute error and mean absolute percentage error; the best model thus sorted out was validated and tested. A multilayer feed forward ANN model with two hidden layers was selected which provided better linear correlation between the chemical composition and impact energy. Correlation performance of considered ANN model with network topology expressed in terms of mean absolute percent error was found to be 0.43% with a correlation coefficient value of 0.95714. Testing and evaluation of the developed model proved to be efficient enough for the development of duplex stainless steels with required impact toughness.

Modeling the Temperature Dependence of Dynamic Mechanical Properties and Visco-Elastic Behavior of Thermoplastic Polyurethane Using Artificial Neural Network.

This paper presents one of the soft computing methods, specifically the artificial neural network technique, that has been used to model the temperature dependence of dynamic mechanical properties and visco-elastic behavior of widely exploited thermoplastic polyurethane over the wide range of temperatures. It is very complex and commonly a highly non-linear problem with no easy analytical methods to predict them directly and accurately in practice. Variations of the storage modulus, loss modulus, and the damping factor with temperature were obtained from the dynamic mechanical analysis tests across transition temperatures at constant single frequency of dynamic mechanical loading. Based on dynamic mechanical analysis experiments, temperature dependent values of both dynamic moduli and damping factor were calculated by three models of well-trained multi-layer feed-forward back-propagation artificial neural network. The excellent agreement between the modeled and experimental data has been found over the entire investigated temperature interval, including all of the observed relaxation transitions. The multi-layer feed-forward back-propagation artificial neural network has been confirmed to be a very effective artificial intelligence tool for the modeling of dynamic mechanical properties and for the prediction of visco-elastic behavior of tested thermoplastic polyurethane in the whole temperature range of its service life.

Modeling suspended sediment concentration using multilayer feedforward artificial neural network at the outlet of the watershed

Modeling suspended sediment concentration using multilayer feedforward artificial neural network at the outlet of the watershed

AN INDOOR POSITIONING TECHNIQUE BASED ON A FEED-FORWARD ARTIFICIAL NEURAL NETWORK USING LEVENBERG-MARQUARDT LEARNING METHOD

Abstract. This paper presents an indoor positioning technique based on a multi-layer feed-forward (MLFF) artificial neural networks (ANN). Most of the indoor received signal strength (RSS)-based WLAN positioning systems use the fingerprinting technique that can be divided into two phases: the offline (calibration) phase and the online (estimation) phase. In this paper, RSSs were collected for all references points in four directions and two periods of time (Morning and Evening). Hence, RSS readings were sampled at a regular time interval and specific orientation at each reference point. The proposed ANN based model used Levenberg–Marquardt algorithm for learning and fitting the network to the training data. This RSS readings in all references points and the known position of these references points was prepared for training phase of the proposed MLFF neural network. Eventually, the average positioning error for this network using 30% check and validation data was computed approximately 2.20 meter.

Estimation of effective connectivity using multi-layer perceptron artificial neural network.

Studies on interactions between brain regions estimate effective connectivity, (usually) based on the causality inferences made on the basis of temporal precedence. In this study, the causal relationship is modeled by a multi-layer perceptron feed-forward artificial neural network, because of the ANN's ability to generate appropriate input-output mapping and to learn from training examples without the need of detailed knowledge of the underlying system. At any time instant, the past samples of data are placed in the network input, and the subsequent values are predicted at its output. To estimate the strength of interactions, the measure of "Causality coefficient" is defined based on the network structure, the connecting weights and the parameters of hidden layer activation function. Simulation analysis demonstrates that the method, called "CREANN" (Causal Relationship Estimation by Artificial Neural Network), can estimate time-invariant and time-varying effective connectivity in terms of MVAR coefficients. The method shows robustness with respect to noise level of data. Furthermore, the estimations are not significantly influenced by the model order (considered time-lag), and the different initial conditions (initial random weights and parameters of the network). CREANN is also applied to EEG data collected during a memory recognition task. The results implicate that it can show changes in the information flow between brain regions, involving in the episodic memory retrieval process. These convincing results emphasize that CREANN can be used as an appropriate method to estimate the causal relationship among brain signals.

Neural network adaptive real-time optimizing control of industrial processes

<p>Real-time optimization (RTO) has attracted considerable interest among researchers and industries for being able to optimise the plant economics such as product efficiency, product quality and process safety in the wake of increasing global competitions. The success of RTO depends much on the quality of model being used in the optimisation. The present study was carried out to explore the use of artificial neural network (ANN) to improve the quality of the model being used in the modified two step (MTS) technique. The MTS is a real-time optimising control algorithm of the modifier adaptation scheme which is used to determine the optimum steady-state control set-points. The proposed new version of MTS technique will be using process model based on ANN. A laboratory scale process of a two continuous stirred tank heat exchanger in series (2CSTHEs) is used as a case study. The multilayer feed forward ANN architecture 4-10-6 with linear function was used to model the 2CSTHEs and then integrates into the MTS technique, the resulted algorithm will be known as Iterative Neural Network Modified Two Step (INNMTS) technique. Simulation studies were conducted to test the performance of the INNMTS technique on the 2CSTHEs process. The results show that the overall value for the coefficient of determination(R<sup>2</sup>)is equal to one, which indicates adequacy of the model proposed for the prediction of the behavior of 2CSTHEs system. When NN model of 2CSTHEs is applied to the INNMTS technique, the model-plant mismatch is greatly reduced to almost zero, which indicates by significant reduction in the number of iterations to 5which requires by INNMTS compared to 16 iterations by the MTS technique to converge to optimal real solution.</p><p>Chemical Engineering Research Bulletin 19(2017) 129-138</p>

Hardware-Efficient On-line Learning through Pipelined Truncated-Error Backpropagation in Binary-State Networks.

Artificial neural networks (ANNs) trained using backpropagation are powerful learning architectures that have achieved state-of-the-art performance in various benchmarks. Significant effort has been devoted to developing custom silicon devices to accelerate inference in ANNs. Accelerating the training phase, however, has attracted relatively little attention. In this paper, we describe a hardware-efficient on-line learning technique for feedforward multi-layer ANNs that is based on pipelined backpropagation. Learning is performed in parallel with inference in the forward pass, removing the need for an explicit backward pass and requiring no extra weight lookup. By using binary state variables in the feedforward network and ternary errors in truncated-error backpropagation, the need for any multiplications in the forward and backward passes is removed, and memory requirements for the pipelining are drastically reduced. Further reduction in addition operations owing to the sparsity in the forward neural and backpropagating error signal paths contributes to highly efficient hardware implementation. For proof-of-concept validation, we demonstrate on-line learning of MNIST handwritten digit classification on a Spartan 6 FPGA interfacing with an external 1Gb DDR2 DRAM, that shows small degradation in test error performance compared to an equivalently sized binary ANN trained off-line using standard back-propagation and exact errors. Our results highlight an attractive synergy between pipelined backpropagation and binary-state networks in substantially reducing computation and memory requirements, making pipelined on-line learning practical in deep networks.

A system for accelerometer-based gesture classification using artificial neural networks.

A great many people suffer from neurological movement disorders that render typical hardware interface devices ineffective. A need exists for a universal interface device that can be trained to accept a wide range of inputs across varying types and severities of movement disorders. In this regard, this paper details the design, testing and optimization of an accelerometer-based gesture identification system. A Bluetooth-enabled IMU mounted on the wrist provides hand motion trajectory information to a local terminal. Several techniques are applied to decrease the intra-class variance and reduce classifier complexity including filtering, segmentation and temporal scaling. Datasets consisted of 520 training samples, 260 validation samples and a further 520 testing samples. A multi-layer feed forward artificial neural network (ML-FFNN) was used to classify the input space into 26 different classes. Initial system accuracy, using arbitrary hyperparameters was 77.69% with final optimized accuracy at 99.42%.

Subsampling-Based Blind Image Forgery Detection Using Support Vector Machine and Artificial Neural Network Classifiers

In order to create convincing doctored images, forged images are exposed to some linear transformations (like rotation and resizing) which involve a resampling step. In copy–paste image forgery, the pasted portion is rescaled in order to hide traces of malicious tampering. In this paper, an algorithm is proposed to detect the global resizing operation of the doctored image blindly based on features extracted using subsampling. Fisher criterion is employed in order to choose the relevant features and reduce the dimensionality of the statistical features. Support vector machine (SVM) and multi-layer feedforward artificial neural network (ANN) are used for classification. Experimental results using $$C_b$$ , $$C_r$$ and grayscale images demonstrate that the proposed method has good rescaling detection performance even when dealing with distortions like JPEG compression. The results indicate that SVM performs better compared to ANN classifier.

Practical modeling and optimization of ultrasound-assisted bleaching of olive oil using hybrid artificial neural network-genetic algorithm technique

Multi-objective modeling and optimization of ultrasound-assisted bleaching of olive oil were accomplished by a hybrid artificial neural network (ANN) and genetic algorithm (GA) method using an ultrasonic bath with a frequency of 25kHz. The influence of process parameters including ultrasonic power, bleaching clay dosage, process temperature and time (inputs) on final Lovibond red (Lr) and peroxide value (PV) (outputs) was modeled by a multilayer feed-forward back propagation ANN. The accurate 2-hidden layer model with 20 neurons in each, high R2 (up to 90%) and minimum mean square error (MSE) obtained by ANN was introduced to GA to find the best operation conditions to achieve minimum Lr and PV. The optimum treatment was found with ultrasonic power of 30%, bleaching clay of 1.2%, bleaching time of 13min and temperature of 65°C. Under optimal conditions, Lr and PV were 2.47 and 6.49 (meqO2/kg), respectively, that were consistent with predicted values.Optimally ultrasonic bleached olive oil and an industrially bleached olive oil were compared. In most cases, the results indicated no detrimental effects of ultrasound on oil structure. Thus, 40% reduction in bleaching clay dosage, 35% reduction in process temperature and 57% reduction in time over ultrasound-assisted bleaching which not only provided economic and environmental benefits, but also retained edible oil nutritional value in comparison to common bleaching procedure. The results of this study confirm the applicability of ultrasound-assisted bleaching by ultrasonic bath as an economic and feasible approach for bleaching of olive oil to reduce high bleaching costs.