Multilayer Feed-forward Back-propagation Neural Network Research Articles

An artificial neural network for predicting air traffic demand based on socio-economic parameters

Over the past five years, emerging economies have consistently progressed toward achieving greater economic independence. The aviation industry has played a significant role in driving this progress. Accurately forecasting the future demand for air travel is crucial for making long-term commitments to these economies. The findings of this study provide valuable insights into the provision of low-cost passenger carrier demand. They can assist low-cost carriers, airports, airport consultants, and government agencies in their strategic planning and development processes. The primary objective of this study is to identify and evaluate artificial intelligence-based solutions, specifically, Artificial Neural Networks (ANN), to determine the most effective model for projecting domestic demand for low-cost carriers. The study analyzes nine variables that impact the accuracy of predicting domestic air passenger demand in emerging countries using a Multilayer Feed-Forward Backpropagation Neural (MFFBPN) network. The performance of the developed ANN model for predicting air passenger demand is assessed through training, validation, and testing using 240 data sets. The evaluation metrics employed, including mean square error (MSE), root mean square error (RMSE), accuracy, precision, sensitivity, and specificity, indicate the effectiveness of the developed ANN model, with results of 0.0019 for MSE, 0.0440 for RMSE, 0.92 for accuracy, 0.92 for precision, 0.93 for sensitivity, and 0.95 for specificity. We show that modeling based on artificial intelligence approaches and ANN is more effective than conventional linear regression models.

Welding sequence optimization to reduce welding distortion based on coupled artificial neural network and swarm intelligence algorithm

This study aims to develop a welding sequence optimization (WSO) framework based on coupled artificial neural network (ANN) and swarm intelligence algorithm for minimizing welding distortion of thin-walled squared Al–Mg–Si alloy tube components. This framework is mainly composed of two critical computer programs. Firstly, a multilayer feedforward backpropagation neural network (BPNN) system was established to rapidly estimate residual distortion for an arbitrary welding sequence so that welding sequence can be optimized for achieving desired welding quality. For this purpose, a series of nonlinear thermo-elastic–plastic finite element (FE) simulations were conducted and verified with experiments to generate the input database of the neural network. Subsequently, a reliable BPNN model was successfully created and trained within an acceptable error. Secondly, a novel swarm intelligence algorithm, namely, bees algorithm (BA) was proposed to solve the complicated WSO problems. In this optimization process, the trained BPNN model was implanted into this proposed BA for computing the fitness value of arbitrary welding sequences. Moreover, welding experiments were also performed to confirm the performance of the proposed optimization method. Comparing the results from experimental measurements, FE simulations, and proposed WSO framework, it is demonstrated that this proposed BPNN-and-BA-based WSO framework can be successfully applied in practical engineering to obtain an optimal welding sequence for minimizing final welding distortion.

Artificial neural networks (ANNs) for modeling efficient factors in predicting pap smear screening behavior change stage.

Background and objectivesCervical cancer is ranked as the third most prevalent cancer that affects women all over the world and Pap smear seems to be the single most critical intervention to prevent cervical cancer. In the present study, the effects of demographic factors (age, education level, job, income level, marriage age, pregnancy, child number, breastfeeding, and menopause), insurance type, disease history and screening (sono and mammography, breast problem) and cancer information on Pap smear screening and behavior stage of change were investigated and modeled using an artificial neural network model (ANN).Materials and methodsData were collected from a descriptive-analytical cross-sectional study. This research was conducted on 1898 female employees of governmental agencies of Birjand, a city located in the east of Iran. The questionnaire consisted of four parts (socioeconomic, reproductive characteristics, information about cervical cancer screening, and stage of change for cervical cancer screening). Multilayer feed-forward back-propagation neural networks were used to detect the patterns between variables using a neural network with 14 inputs and one output. To find out the neural network with the minimum sum of squared errors, we evaluated the performance of all neural networks using varying algorithms and numbers of neurons in the hidden layer. For this purpose, the data collected from 1898 women were analyzed using SPSS-22 software.ResultsIn the optimal ANN model, the variables of marriage age, age, breastfeeding, and the number of children were identified as the most significant factors with 18.3, 16.3, 7.3, and 7.3 percent, respectively, whereas the history of cancer, job, pregnancy, and menopause had importance of lower than 5 percent.ConclusionOur findings showed that among many associated variables, the marriage age, age, breastfeeding, and the number of children were the most important predictors for the behavioral stage of change. Thus, it seems, focusing on these factors may lead to the adoption of effective programs and policies to improve cervical cancer screening practices in women.

Applying artificial neural-network model to predict psychiatric symptoms.

IntroductionMental disorders result in mental disabilities and discomfort in the affected person as they affect both thinking and behavior. Therefore, more vulnerable people must first be identified to improve the psychological level of society.AimThis study aims to determine the importance of gender, employment, education, place of residence, and age in predicting mental disorders using artificial neural network modeling.MethodsThe pattern held between variables in this study will be identified using multilayer feed-forward back-propagation neural networks with five inputs and 10 outputs. To determine the neural network with the least sum of square errors, we evaluated the performance of all neural networks with varying algorithms and different numbers of neurons in the hidden layer. Data were analyzed for 380 people aged 10–82 years using the SPSS software.ResultsThe optimal neural network model was effective in predicting mental disorders. In this model, the variables of the place of residence, education, age, gender, and employment were important in fitting the optimal model with 34.08, 20.11, 18.93, 14.55, and 12.33%, respectively. The accuracy rate for the neural network model was 99.2%.ConclusionTo achieve further results in improving mental health problems, it is better to focus more on employed, rural, and younger people with a non-tertiary education level.

Critical evaluation and comparison of psychoacoustics, acoustics and vibration features for gear fault correlation and classification

Gear fault diagnosis has gained importance in the last few decades with the focus of fault diagnosis function for maintenance purpose. This paper investigates the ability of various vibration, acoustics and psychoacoustic features to correlate and classify faults in the gearbox. This study is motivated by the process of gearbox fault diagnosis at the end of the assembly line inspection of gearbox. It is observed that during the end of assembly line inspection, the gearbox is operated on a test bench and operator takes a decision about the presence of fault by listening to the sound emitted by the gearbox. This decision is based on an operator’s judgement and past experience which involves subjectivity. Efforts are made to address this issue of end of the line inspection, by applying a scientific, objective psychoacoustic based technique which works on similar principles of listening. Experiments are performed in a laboratory by simulating four types of gear faults and vibration and acoustic signals are acquired to extract statistical features of vibration and acoustic signal along with psychoacoustic features. It is observed that different features respond to faults in different ways and changes in feature values are also dependent on loading condition. Therefore, correlating these features with faults is not simple and need intelligent techniques for correlating faults to features. This correlation of faults to features is attempted with different techniques like ANN and discriminant classifier. It is found that the psychoacoustic features have better ability to classify faults compared to acoustics and vibration statistical features which are found to be 93.02% using quadratic discriminant classifier and 95.93% using multilayer feed-forward back-propagation neural network. It is shown that the psychoacoustic based fault identification technique can be applied for the end of the assembly line inspection of gearbox.

Machine learning model for dynamical response of nano-composite pipe conveying fluid under seismic loading

Machine learning approaches including support vector regression (SVR) and multi-layer feedforward backpropagation neural network (FFBNN) were used in the present study along with classic theory for predicting maximum displacement of nanocomposite pipe conveying fluid under seismic load. The FFBNN consisted of three layers: 1) three neurons in input layer including length-to-radius ratio (L/R), fluid velocity (V) and volume percent of carbon nanotube; 2) hidden layer with 11 neurons obtained via trial and error; 3) maximum displacement-based seismic load. SVR model was obtained via three-input data with maximum likelihood estimator. Model predicted results were compared using three metrics, including Nash-Sutcliffe efficiency, root mean squared error and coefficient of correlation for 100 testing and 255 training data points. Results indicated that SVR achieved best predictions in the training phase, while FFBNN provided superior prediction in the testing phase. Increasing L/R, V and decreasing VCNT, increased maximum displacements under seismic load.

Machine Learning Model for Dynamical Response of Nano-composite Pipe Conveying Fluid under Seismic Loading

Machine learning approaches including support vector regression (SVR) and multi-layer feedforward backpropagation neural network (FFBNN) were used in the present study along with classic theory for predicting maximum displacement of nanocomposite pipe conveying fluid under seismic load. The FFBNN consisted of three layers: 1) three neurons in input layer including length-to-radius ratio (L/R), fluid velocity (V) and volume percent of carbon nanotube; 2) hidden layer with 11 neurons obtained via trial and error; 3) maximum displacement-based seismic load. SVR model was obtained via three-input data with maximum likelihood estimator. Model predicted results were compared using three metrics, including Nash-Sutcliffe efficiency, root mean squared error and coefficient of correlation for 100 testing and 255 training data points. Results indicated that SVR achieved best predictions in the training phase, while FFBNN provided superior prediction in the testing phase. Increasing L/R, V and decreasing VCNT, increased maximum displacements under seismic load.

The effect on the wind power performance of different normalization methods by using multilayer feed-forward backpropagation neural network

Artificial Neural Networks is the most used machine learning approach today. It is a very successful method in terms of accuracy and reliability. It is widely used in classification and estimation calculations. In order to achieve the desired performance a model created with ANN, a series of processes such as selection of network structure, learning algorithms, input and output values adjustment and transfer functions determination needs to be implemented in a sensitive manner. Multilayer Feedforward Backpropagation Network, which is used most frequently in supervised learning approaches, was considered in this study. The effect on the prediction performance of the developed model was investigated by using different statistical normalization methods on the data to be used in the network. For this purpose, 4-input 1-output artificial neural networks model were operated with wind-based data taken from Osmaniye Korkut Ata University measuring station. Wind speed, Wind Direction, Humidity and Density data are defined as input values while wind power was defined as output value. Input and output data are calculated with different normalization methods and more than one network models are designed with calculated values. As a result, the study showed that artificial neural networks model which is established by sigmoid normalization method has the best performance value.

A scalable hybrid decision system (HDS) for Roman word recognition using ANN SVM: study case on Malay word recognition

An off-line handwriting recognition (OFHR) system is a computerized system that is capable of intelligently converting human handwritten data extracted from scanned paper documents into an equivalent text format. This paper studies a proposed OFHR for Malaysian bank cheques written in the Malay language. The proposed system comprised of three components, namely a character recognition system (CRS), a hybrid decision system and lexical word classification system. Two types of feature extraction techniques have been used in the system, namely statistical and geometrical. Experiments show that the statistical feature is reliable, accessible and offers results that are more accurate. The CRS in this system was implemented using two individual classifiers, namely an adaptive multilayer feed-forward back-propagation neural network and support vector machine. The results of this study are very promising and could generalize to the entire Malay lexical dictionary in future work toward scaled-up applications.

Integration of artificial neural network and geographical information system for intelligent assessment of land suitability for the cultivation of a selected crop

The main objective of this study is to investigate the potential of artificial neural network (ANN) for integration with geographical information system (GIS) to assess the suitability of land to cultivate a selected crop. For this purpose, requirements of a system for intelligent assessment of land suitably were determined and the architecture of the integrated system was designed according to capabilities of ANN and GIS. For evaluation of the integrated system performance, a case study was carried out in the Mazandaran Province, located in the northern part of Iran. In the system implemented for the case study, four effective parameters in crop cultivation were utilized in the multilayer feed-forward back-propagation neural network analysis. The activation function of the hidden layer was set to log-sigmoid function. Designed ANN was trained using the samples extracted from GIS database, and 4-6-1 (4 input parameters as input layer, 6 neurons in hidden layer and 1 target value representing suitability level) structure shows the best result. Evaluation of the system shows that 83.43 % of ANN's results are acceptable and consistent with the real world.

The application of multilayer perceptron neural network in volume of clay estimation: Case study of Shurijeh gas reservoir, Northeastern Iran

Volume of clay is an important component in the assessment of shaly sand reservoirs, due to its significant impact on the production characteristics. The Shurijeh sandstone Formation of Lower Cretaceous age, with subordinate shales is one of the most challenging gas reservoirs to be properly characterized in the eastern Kopet-Dagh sedimentary Basin, Northeastern Iran. This paper describes the improvement achieved in estimating the volume of clay in the Shurijeh reservoir Formation, with an application to a gas producing well and another non-producing well in a joint field between Iran and Turkmenistan. A clear comparison between estimates from several conventional petrophysical methods and actual laboratory measured data of 76 core samples showed very large estimation errors; therefore an attempt has been made to improve the estimation with developing a multilayer feedforward backpropagation neural network. Six types of well logs were selected, through a sensitivity analysis, as the most relevant input data to the volume of clay (network output). Data were then standardized and randomly divided into three sets of 70% for training, 15% for validation and 15% for testing. Three different training algorithms of genetics, particle swarm optimization, and Levenberg–Marquardt were tested on a network with certain topology, and the latter was chosen, due to the better performance. The number of hidden layer neurons, was efficiently determined 8 through a trial and error process. The developed network with the architecture of 6-8-1 was then successfully validated with 16 unseen core data. Mean squared error of 2.8069 E−3 and a correlation coefficient of 0.9013, as the network criteria; clearly showed a multilayer neural network can significantly improve the estimation of volume of clay by 53% within data set from the Shurijeh Formation.

Wind Power Forecasting in a Residential Location as Part of the Energy Box Management Decision Tool

A number of location characteristics, such as buildings, mountains, and trees, are likely to influence the wind flow that reaches a microwind turbine located at a residential area, and as a consequence they may affect the actual wind speed that is potentially utilized by the turbine. In this context, simple regional predictions for the wind energy from the nearest location available can easily lead to unacceptable modeling errors. There is thus a need to develop a framework for predicting the values of the wind speed at the desired location. This work addresses the aforementioned issue by employing a multilayer feed-forward back-propagation neural network for classification that utilizes the global forecast system (GFS) predictions on wind speed and direction to identify patterns of the wind behavior at the location considered in order to obtain a stochastic distribution of the daily wind speed. The proposed approach aims to support the implementation of an enhanced energy box (EB) management decision tool, while its feasibility is demonstrated through a case example for a region in the south of Portugal.

Multi-criteria inventory classification through integration of fuzzy analytic hierarchy process and artificial neural network

A systematic approach to the inventory control and classification may have a significant influence on company competitiveness. In practice, all inventories cannot be controlled with equal attention. To efficiently control the inventory items and to determine the suitable ordering policies for them, multi- criteria inventory classification is used. The objective of this research is to develop a multi-criteria inventory classification model through integration of fuzzy analytic hierarchy process (FAHP) and artificial neural network approach. FAHP is used to determine the relative weights of the attributes or criteria using Chang's extent analysis and to classify inventories into different categories. Various structures of multi-layer feed-forward back-propagation neural networks have been analysed and the optimal one with the minimum mean absolute percentage of error between the measured and the predicted values have been selected. To accredit the proposed model, it is implemented for 351 raw materials of switchgear section of Energypac Engineering Limited, a large power engineering company of Bangladesh.

Development of a License Plate Recognition System for a Non-ideal Environment

A new algorithm for license plate character recognition system is proposed on the basis of Signature analysis properties and features extraction. Signature analysis has been used to locate license plate region and its properties can be further utilised in supporting and affirming the license plate character recognition. This paper presents the implementation of Signature Analysis combined with Features Extraction to form feature vector for each character with a length of 56. Implementation of these two methods is used in tracking of vehicle’s automatic license plate recognition system (ALPR). The developed ALPR comprises of three phase. The recognition stage utilised the vector to be trained in a simple multi-layer feed-forward back-propagation Neural Network with 56 inputs and 34 neurons in its output layer. The network is trained with both ideal and noisy characters. The results obtained show that the proposed system is capable to recognise both ideal and non-ideal license plate characters. The system also capable to tackle the common character misclassification problems due to similarity in characters.

An artificial immune network approach to multi-sensor land use/land cover classification

An optimized artificial immune network-based classification model, namely OPTINC, was developed for remote sensing-based land use/land cover (LULC) classification. Major improvements of OPTINC compared to a typical immune network-based classification model (aiNet) include (1) preservation of the best antibodies of each land cover class from the antibody population suppression, which ensures that each land cover class is represented by at least one antibody; (2) mutation rates being self-adaptive according to the model performance between training generations, which improves the model convergence; and (3) incorporation of both Euclidean distance and spectral angle mapping distance to measure affinity between two feature vectors using a genetic algorithm-based optimization, which helps the model to better discriminate LULC classes with similar characteristics. OPTINC was evaluated using two sites with different remote sensing data: a residential area in Denver, CO with high-spatial resolution QuickBird image and LiDAR data, and a suburban area in Monticello, UT with HyMap hyperspectral imagery. A decision tree, a multilayer feed-forward back-propagation neural network, and aiNet were also tested for comparison. Classification accuracy, local homogeneity of classified images, and model sensitivity to training sample size were examined. OPTINC outperformed the other models with higher accuracy and more spatially cohesive land cover classes with limited salt-and-pepper noise. OPTINC was relatively less sensitive to training sample size than the neural network, followed by the decision tree.

Predicting effect of interparticle interactions on permeate flux decline in CMF of colloidal suspensions: An overlapped type of local neural network

An overlapped type of local neural network is proposed to improve accuracy of the flux decline prediction in crossflow membrane filtration (CMF) of colloidal suspensions. This network combines the advantages of the multilayer feedforward back-propagation neural network and the radial basis function network. The prediction performance and efficiency of the proposed network are examined using a published experimental dataset of CMF. The data includes the permeate flux decline under hydrodynamic and physicochemical operating conditions, such as transmembrane pressure, particle size, ionic strength and solution pH. Unlike the conventional network using the trial-and-error method, the proposed network offers a systematical framework for constructing and training network based on the data distribution of the flux decline in CMF, bringing the benefits of fast convergence and easy training. The results indicate that the proposed network can find out the proper locations of the neurons to improve the approximation without trial-and-error procedures. The effective contribution of the colloidal interactions, hydrodynamic forces and the coupling between these parameters on permeate flux decline is further computed based on all flux experiments for the whole period of measurement. Because of the good initialization of the neural network, the computed effect of the operation condition extracting from the same set of network connection weights can be kept consistent. Thus, the validation results of the cause-effect information quantitatively extracted from the network match our expectation of the coupling effect on permeate flux decline in CMF at any time.

Predicting flux decline in crossflow membranes using artificial neural networks and genetic algorithms

The geometry and internal parameters of artificial neural networks (ANNs) have significant effects on the prediction performance efficiency of the network. The optimal ANN geometry is problem-dependent. Although some guidance is available in the literature for the choice of geometry and internal parameters, most networks are calibrated using the trial-and-error approach. This paper presents the use of genetic algorithms (GAs) to search the optimal geometry and values of internal parameter of a multilayer feedforward back-propagation neural network (BPNN) and a radial basis function network (RBFN). The prediction performance efficiency of the GA–ANN combination is examined using an already published experimental dataset of crossflow membrane filtration. The data includes the permeate flux decline under various operating conditions (e.g. transmembrane pressure and filtration time) with different physicochemical properties of feed water (e.g. different combinations of three particle diameters, three pH values and four ionic strengths). It is illustrated that the GA-optimized ANN predicts the permeate flux decline more accurately than a network in which the ANN calibration is done using a trial-and-error approach. It is shown that scaling the training data to the range of 0–1 helps the modeler find the solution range of an RBFN for GA.

Prediction of permeate flux decline in crossflow membrane filtration of colloidal suspension: a radial basis function neural network approach

The capability of a radial basis function neural network (RBFNN) to predict long-term permeate flux decline in crossflow membrane filtration was investigated. Operating conditions of transmembrane pressure and filtration time along with feed water parameters such as particle radius, solution pH, and ionic strength were used as inputs to predict the permeate flux. Simulation results indicated that a single RBFNN accurately predicted the permeate flux decline under various experimental conditions of colloidal membrane filtrations and eventually produced better predictability than those of the regular multi-layer feed-forward backpropagation neural network (BPNN) and the multiple regression (MR) method. We believe further development of the artificial neural network approach will enable us to design and analyze full-scale processes from results of laboratory and/or pilot-scale experiments.

Improving the inversion of ionograms by combining neural network and data fusion techniques

Data fusion (integration) techniques are combined with MultiLayer Feedforward (backpropagation) neural networks in order to improve the inversion —extraction of the key describing parameters — of oblique-incidence ionograms (plots of apparent height of reflection versus transmission frequency). Two separate investigations were undertaken: first, the incorporation of vertical ionogram data to improve inversion; and secondly, the fusion of ionogram data gathered from a 2D array of ionosondes (the ground-based radio frequency transmitters). With the former, the average percentage errors obtained by incorporating data fusion dropped by a factor of five when compared with single ionogram inversion. Moreover, gradients of ionospheric parameters (critical frequency, layer height and thickness) were also obtained. In the case of the latter, the error rate dropped by a similar factor, and by even more when vertical ionograms were incorporated. Better results were forthcoming when a hierarchical network was used to invert the ionograms prior to fusion, compared with directly fusing the ionogram array data.

Pruning Divide & Conquer networks

Determining an effective architecture for multi-layer feedforward backpropagation neural networks can be a time-consuming effort. In general it requires human intervention in determining the number of layers, number of hidden cells, the learning rule and the learning parameters. Over the past few years several approaches to dynamically configure neural networks have been proposed, which remove most of the responsibility for choosing the correct network configuration from the user. As important as finding a viable network architecture for some given learning problem, is the need to obtain a minimal configuration. The total time required to emulate or simulate neural networks is largely dependent on the number of connections present in a network. Therefore, it is essential to provide pruning methods to reduce network complexity. In this paper two approaches to network pruning will be investigated: single- and multi-pass pruning. Their effectiveness is emphasized by applying them to several real world problem domains and comparing them to a modification of Optimal Brain Damage. It will be shown that multi-pass pruning is not only fast, but also effective in reducing the network size when used in conjunction with Divide & Conquer Learning. Finally, it is general enough to be applied to other dynamically network configuring learning approaches.