Back-propagation Multilayer Network Research Articles

Estimating the Unit Weight of Local Organic Soils from Laboratory Tests Using Artificial Neural Networks

The estimation of the unit weight of soil is carried out using laboratory methods; however, it requires high-quality research material in the form of samples with undisturbed structures, the acquisition of which, especially in the case of organic soils, is extremely difficult, time-consuming and expensive. This paper presents a proposal to use artificial neural networks to estimate the unit weight of local organic soils as leading parameters in the process of checking the load capacity of subsoil, under a direct foundation in drained conditions, in accordance with current standards guidelines. The initial recognition of the subsoil, and the locating of organic soils at the Theological and Pastoral Institute in Rzeszow, was carried out using a mechanical cone penetration test (CPTM), using various interpretation criteria, and then, material for laboratory tests was obtained. The analysis of the usefulness of the artificial intelligence method, in this case, was based on data from laboratory tests. Standard multi-layer backpropagation networks were used to predict the soil unit weight based on two leading variables: the organic content LOIT and the natural water content w. The applied neural model provided reliable prediction results, comparable to the standard regression methods.

Machine learning based ovarian detection in ultrasound images

Computer aided ovarian detection and ovarian classification is important in infertility treatment in women. In the proposed methodology, an intelligent automatic detection and ovarian classification with grading based on integration of intensity and texture features using artificial neural network is developed. Three texture features such as autocorrelation, sum average and sum variance obtained from grey-level co-occurrence matrix (GLCM) and intensity obtained using K-means clustering were fed as input to the multilayer feedforward backpropagation network for ovarian detection. Ovarian morphology was used for classification and grading of ovary. This novel technique helps the physician to grade the follicle/cyst. Performance metrics like sensitivity, specificity, accuracy, precision, F-measure, Mathew's correlation coefficient and receiver operating characteristic curve were used to prove the effectiveness of the proposed machine learning based ovarian detection (MLOD). The MLOD classifier yielded an average detection accuracy of 96% which is an increase of 2% as compared to the combined texture and intensity based ovarian classification (TIOC) algorithm.

Prediction of the Superficial Heat Source Parameters for TIG Heating Process Using FEM and ANN Modeling

The basic problem of the numerical model’s quenching process is establishing the characteristics of the boundary conditions. The existing descriptions of the boundary conditions, which represent the parameters of equipment used in heat treatment processes, do not accurately reflect the actual process conditions. In the present study, the method of choice for superficial heat source parameters for TIG (tungsten inert gas) heating is modeled using artificial neural networks (ANN) and the finite element method (FEM). A comparison of the calculations obtained from the numerical model of non-steady state heat transfer with the results of the experimental studies is presented. The possibility of using ANN to compute the parameters of the boundary conditions for the heating treatment is analyzed. A multilayer feed-forward backpropagation network is developed and trained using value of temperature in the selected nodes obtained from numerical simulation.

Can machine learning approaches predict corporate bankruptcy? Evidence from a qualitative experimental design

Bankruptcy prediction has received a growing interest in corporate finance and risk management recently. Although numerous studies in the literature have dealt with various statistical and artificial intelligence classifiers, their performance in credit risk forecasting needs to be further scrutinized compared to other methods. In the spirit of Chen, Härdle and Moro (2011, Quantitative Finance), we design an empirical study to assess the effectiveness of various machine learning topologies trained with big data approaches and qualitative, rather than quantitative, information as input variables. The experimental results from a ten-fold cross-validation methodology demonstrate that a generalized regression neural topology yields an accuracy measurement of 99.96%, a sensitivity measure of 99.91% and specificity of 100%. Indeed, this specific model outperformed multi-layer back-propagation networks, probabilistic neural networks, radial basis functions and regression trees, as well as other advanced classifiers. The utilization of advanced nonlinear classifiers based on big data methodologies and machine learning training generates outperforming results compared to traditional methods for bankruptcy forecasting and risk measurement.

Backpropagation to predict drinking water consumption

The present research article is based on the use of artificial neural networks as a tool for the prediction applied to the consumption of drinking water, where the artificial learning of a multilayer backpropagation network with the historical data consumed in m3 is used. In a computer with a regular feature, the backpropagation network was implemented in the Python 3.7.0 language, taking as a study object a case published on its SUNEDU website regarding drinking water consumption in 2017 and checking the results in January of 2018, taking these data for learning and the respective tests. It has been achieved as a result to predict the amount of water consumption of the institution. The test performed resulted in an excess of 23 m3, which represents 2.7% in excess given that in January 2018 a consumption of 833m3 was recorded. Bear in mind that it was done for the training process with a maximum error for the training of 0.000099 and with a maximum number of iterations of 100000. After training with the historical data of drinking water consumption, it was possible to predict the consumption of SUNEDU to January 2018.

Comparing the Forecasting Performance of Futures Oil Prices with Genetically Evolved Neural Networks

In this paper, a hybrid system combining neural networks and genetic training is designed to forecast future oil prices. The architectural design is that of the multilayer back propagation network that is fed monthly prices for West Texas Intermediate covering the period 1986–2014. The model’s predictions are compared to those of the one, two, three, and four-month futures prices and are evaluated both on their level of accuracy as well as correctness. While accuracy measures the degree of error, correctness tests the model’s ability to predict the direction of the movement. By processing information more efficiently, and identifying patterns that may be ill-defined as a result of pronounced price volatility, this paper aims to improve the accuracy of oil price forecasts.

A Bayesian regularized artificial neural network for adaptive optics forecasting

Real-time adaptive optics is a technology for enhancing the resolution of ground-based optical telescopes and overcoming the disturbance of atmospheric turbulence. The performance of the system is limited by delay errors induced by the servo system and photoelectrons noise of wavefront sensor. In order to cut these delay errors, this paper proposes a novel model to forecast the future control voltages of the deformable mirror. The predictive model is constructed by a multi-layered back propagation network with Bayesian regularization (BRBP). For the purpose of parallel computation and less disturbance, we adopt a number of sub-BP neural networks to substitute the whole network. The Bayesian regularized network assigns a probability to the network weights, allowing the network to automatically and optimally penalize excessively complex models. The simulation results show that the BRBP introduces smaller mean absolute percentage error (MAPE) and mean square errors (MSE) than other typical algorithms. Meanwhile, real data analysis results show that the BRBP model has strong generalization capability and parallelism.

Automatic diagnosis of alcohol use disorder using EEG features

Alcohol use disorder (AUD) has been considered as a social and health issue worldwide. More importantly, the screening of AUD patients has been challenging due to the subjectivity imparted by self-test reports. Automated methods involving neuroimaging modality such as quantitative electroencephalography (QEEG) have shown promising research results. However, the QEEG methods were developed only for alcohol dependents (AD) and healthy controls. Therefore, this study sought to propose a machine learning (ML) method to classify 1) between alcohol abusers and healthy controls, and 2) among healthy controls, alcohol abusers, and alcoholics. The proposed ML method involved QEEG feature extraction, selection of most relevant features, and classification of the study participants into their relevant groups. The study participants such as 12 alcohol abusers (mean age 56.70±15.33 years), 18 alcoholics (mean age 46.80±9.29 years), and 15 healthy controls (mean 42.67±15.90 years) were recruited to acquire EEG data. The data were recorded during 10 minutes of eyes closed (EC) and eyes open (EO) conditions. Furthermore, the EEG data were utilized to extract QEEG features such as absolute power (AP) and relative power (RP). Methods such as t-test and principal component analysis (PCA) were employed to select most relevant QEEG features. Finally, the discriminant QEEG features were used as inputs to the classification models: Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Multilayer back-Propagation Network (MLP), and Logistic Model Trees (LMT), supported by 10-fold cross validation. As results, the LMT has achieved best performance rendering a classification accuracy (96%), sensitivity (97%) and specificity (93%). In addition, a further classification for each subgroup of AUD patients has achieved accuracy (>90%). In conclusion, the results implicated significant neurophysiological differences among alcohol abusers, alcoholics, and controls. Moreover, the AUD patients exhibited significantly decreased theta as compared with the healthy controls.

Development of correlations and artificial neural network models to predict second-degree burn time for thermal-protective fabrics

Firefighters and operators working near industrial furnaces or ovens usually encounter extreme environmental conditions. Heat transfer through fabrics exposed to such high heat flux exposures causes skin burn. There are various parameters which govern heat transfer and second degree burn time in such situations. In this work, six dimensionless parameters associated with heat transfer through fabrics exposed to various flame and radiant heat flux exposures are obtained. Correlations for dimensionless second degree burn time are obtained in terms of other non-dimensional parameters. These separate correlations for each type of exposure can be used to obtain thermal protective performance (TPP) of any fabric exposed to different type of heat conditions, provided that fabric material, fabric geometric parameters, air gap between fabric and skin, and intensity of heat exposure are known. Artificial neural network (ANN) approach is also used to predict TPP of clothing. Multi-layer feed-forward back-propagation networks are developed for different type of exposures. Results shows that both correlations and ANN approach used in the study are promising. ANN model predictions are found to agree better with experimental observations as compared to the outcome of developed correlations. Importance of various dimensionless parameters obtained using dimensional analysis are also emphasized.

Modelling of a coagulation chemical dosing unit for water treatment plants using fuzzy inference system

In this study, adaptive neuro-fuzzy inference system (ANFIS) was applied to estimate the parameters of a coagulation chemical dosing unit for water treatment plants. The dosing unit has three input variables (sudfloc 3835, ferric chloride and hydrated lime flow rates) and two output variables (surface charge and pH values). The ANFIS model is compared with multilayer backpropagation network (MBPN) with four different training algorithms for performance evaluation purpose. The results of evaluation tests using the average percentage error (APE), root mean squared error (RMSE), correlation coefficient (R) and average relative variance (ARV) criteria show that ANFIS is the most efficient and reliable estimator when the models were presented with noiseless and noisy input datasets.

Combining neural networks and ANFIS classifiers for supervised examining of electrocardiogram beats

In this paper, the supervised classification of the electrocardiogram (ECG) beats based on the fusion of several intelligent learning machines is described. For classification of ECG heartbeats, first, the QRS complexes are delineated by an efficient algorithm so as to identify the fiducial and J-locations of each complex. For each delineated QRS complex, a feature vector is established based on the geometrical properties of the complex waveform and its associated discrete-wavelet transform. Next, three different multi-layer perceptron back-propagation (MLP-BP) networks are trained with different topologies and intrinsic parameters. Afterwards, the outputs of MLP-BPs are used as the new feature space elements for training three adaptive fuzzy network inference systems (ANFIS) in order to increase the final accuracy. At the end, the outputs of ANFIS classifiers are voted based on majority for each input sample. The method was applied to seven arrhythmias (Normal, LBBB, RBBB, PVC, APB, VE, VF) which belong to the MIT-BIH Arrhythmia Database and the average accuracy value Acc = 98.28% was achieved for the beat-level. Also, the proposed method was assessed to five arrhythmias (Normal, LBBB, RBBB, PVC, APB) according to validation standards of the American Heart Association (AHA) at record (subject) level and the average accuracy value Acc = 73.39% was achieved. To evaluate performance quality of the new proposed hybrid learning machine, the obtained results were compared with similar peer-reviewed studies in this area.

Medical Decision-Making System of Ultrasound Carotid Artery Intima–Media Thickness Using Neural Networks

The objective of this work is to develop and implement a medical decision-making system for an automated diagnosis and classification of ultrasound carotid artery images. The proposed method categorizes the subjects into normal, cerebrovascular, and cardiovascular diseases. Two contours are extracted for each and every preprocessed ultrasound carotid artery image. Two types of contour extraction techniques and multilayer back propagation network (MBPN) system have been developed for classifying carotid artery categories. The results obtained show that MBPN system provides higher classification efficiency, with minimum training and testing time. The outputs of decision support system are validated with medical expert to measure the actual efficiency. MBPN system with contour extraction algorithms and preprocessing scheme helps in developing medical decision-making system for ultrasound carotid artery images. It can be used as secondary observer in clinical decision making.

Intelligent Medical Decision System for Identifying Ultrasound Carotid Artery Images with Vascular Disease

The objective of this work is to develop and implement an intelligent medical decision system for identifying Ultrasound (US) carotid artery images with vascular diseases. The proposed method categorizes the carotid artery subjects into normal and diseased subjects’ namely cerebrovascular and cardiovascular diseases. For each and every preprocessed ultrasound carotid artery image, contours are extracted using contour extraction techniques. Multilayer Back Propagation Network (MBPN) system has been developed for categorizing the carotid artery subjects. The obtained results show that MBPN system provides higher classification efficiency, with minimum training and testing time. It helps in developing Medical Decision System (MDS) for ultrasound carotid artery images. It can also be used as secondary observer in clinical decision making.

A Hybrid Fuzzy-Neural System for Computer-Aided Diagnosis of Ultrasound Kidney Images Using Prominent Features

The objective of this work is to develop and implement a computer-aided decision support system for an automated diagnosis and classification of ultrasound kidney images. The proposed method distinguishes three kidney categories namely normal, medical renal diseases and cortical cyst. For the each pre-processed ultrasound kidney image, 36 features are extracted. Two types of decision support systems, optimized multi-layer back propagation network and hybrid fuzzy-neural system have been developed with these features for classifying the kidney categories. The performance of the hybrid fuzzy-neural system is compared with the optimized multi-layer back propagation network in terms of classification efficiency, training and testing time. The results obtained show that fuzzy-neural system provides higher classification efficiency with minimum training and testing time. It has also been found that instead of using all 36 features, ranking the features enhance classification efficiency. The outputs of the decision support systems are validated with medical expert to measure the actual efficiency. The overall discriminating capability of the systems is accessed with performance evaluation measure, f-score. It has been observed that the performance of fuzzy-neural system is superior compared to optimized multi-layer back propagation network. Such hybrid fuzzy-neural system with feature extraction algorithms and pre-processing scheme helps in developing computer-aided diagnosis system for ultrasound kidney images and can be used as a secondary observer in clinical decision making.

Artificial neural network prediction on wear properties of high vanadium high speed steel (HVHSS) rolls

The present paper is dedicated to the application of artificial neural networks in the prediction of the wear properties of high vanadium high speed steel (HVHSS) rolls, including predictions of wear weight loss W according to carbon content C and number of revolutions N. Multilayer backpropagation networks were created and trained using comprehensive datasets tested by the authors. Very good performances of the neural networks were achieved. The prediction results show that the wear weight loss nearly linear increases with increasing number of revolutions at constant carbon content. The relative wear resistance of roll reaches the optimal value when the carbon content is ∼2·55 wt-%. The prediction values have sufficiently mined the basic domain knowledge of wear process of HVHSS rolls. A convenient and powerful method of optimising the process parameters of abrasive resistant materials has been provided by the authors.

Optimization of heat treatment technique of high-vanadium high-speed steel based on back-propagation neural networks

This paper is dedicated to the application of artificial neural networks in optimizing heat treatment technique of high-vanadium high-speed steel (HVHSS), including predictions of retained austenite content ( A), hardness ( H) and wear resistance ( ε) according to quenching and tempering temperatures ( T1, T2). Multilayer back-propagation (BP) networks are created and trained using comprehensive datasets tested by the authors. And very good performances of the neural networks are achieved. The prediction results show residual austenite content decreases with decreasing quenching temperature or increasing tempering temperature. The maximum value of relative wear resistance occurs at quenching of 1000–1050 °C and tempering of 530–560 °C, corresponding to the peak value of hardness and retained austenite content of about 20–40 vol%. The prediction values have sufficiently mined the basic domain knowledge of heat treatment process of HVHSS. A convenient and powerful method of optimizing heat treatment technique has been provided by the authors.

Predictive models in ecology: Comparison of performances and assessment of applicability

Ecological systems are governed by complex interactions which are mainly nonlinear. In order to capture the inherent complexity and nonlinearity of ecological, and in general biological systems, empirical models recently gained popularity. However, although these models, particularly connectionist approaches such as multilayered backpropagation networks, are commonly applied as predictive models in ecology to a wide variety of ecosystems and questions, there are no studies to date aiming to assess the performance, both in terms of data fitting and generalizability, and applicability of empirical models in ecology. Our aim is hence to provide an overview for nature of the wide range of the data sets and predictive variables, from both aquatic and terrestrial ecosystems with different scales of time-dependent dynamics, and the applicability and robustness of predictive modeling methods on such data sets by comparing different empirical modeling approaches. The models used in this study range from predicting the occurrence of submerged plants in shallow lakes to predicting nest occurrence of bird species from environmental variables and satellite images. The methods considered include k-nearest neighbor (k-NN), linear and quadratic discriminant analysis (LDA and QDA), generalized linear models (GLM) feedforward multilayer backpropagation networks and pseudo-supervised network ARTMAP. Our results show that the predictive performances of the models on training data could be misleading, and one should consider the predictive performance of a given model on an independent test set for assessing its predictive power. Moreover, our results suggest that for ecosystems involving time-dependent dynamics and periodicities whose frequency are possibly less than the time scale of the data considered, GLM and connectionist neural network models appear to be most suitable and robust, provided that a predictive variable reflecting these time-dependent dynamics included in the model either implicitly or explicitly. For spatial data, which does not include any time-dependence comparable to the time scale covered by the data, on the other hand, neighborhood based methods such as k-NN and ARTMAP proved to be more robust than other methods considered in this study. In addition, for predictive modeling purposes, first a suitable, computationally inexpensive method should be applied to the problem at hand a good predictive performance of which would render the computational cost and efforts associated with complex variants unnecessary.

A neural network model for speech intelligibility quantification

A neural network based model is developed to quantify speech intelligibility by blind-estimating speech transmission index, an objective rating index for speech intelligibility of transmission channels, from transmitted speech signals without resort to knowledge of original speech signals. It consists of a Hilbert transform processor for speech envelope detection, a Welch average periodogram algorithm for envelope spectrum estimation, a principal components analysis (PCA) network for speech feature extraction and a multi-layer back-propagation network for non-linear mapping and case generalisation. The developed model circumvents the use of artificial test signals by exploiting naturally occurring speech signals as probe stimuli, reduces measurement channels from two to one and hence facilitates in situ assessment of speech intelligibility. From a cognitive science viewpoint, the proposed method might be viewed as a successful paradigm of mimicking human perception of speech intelligibility using a hybrid model built around artificial neural networks.

Are multi-layer backpropagation networks catastrophically amnesic?

Connectionist models with a backpropagation learning rule are known to have a serious problem. Such models exhibit catastrophic interference (or forgetting) with sequential training. Having learned a set of patterns, if the model is trained on another set of patterns, its performance on the first set can dramatically deteriorate very rapidly. The present study reconsiders this issue with four simulations. The model learned arithmetic facts sequentially, but the interference was only modest with random (hence approximately orthogonal) inputs. Essentially the same result was obtained when the inputs are made less orthogonal by adding irrelevant elements. Reducing the number of hidden units did not have major effects. This study suggests that the interference problem has been somewhat overstated.

Fitting transducer characteristics to measured data using a virtual curve tracer

Abstract This paper presents an artificial neural network based approach of transducer modeling in a computer-based measurement system. Transducer’s static response characteristic is generally highly non-linear. However, if linear approximations were acceptable, for a given accuracy level, noise and measurement errors are always present. Therefore, curve-fitting procedures are required frequently to average these effects. With the facility of computation now available through digital computers and microprocessors, the problem of estimation of transducer’s input–output static response characteristics is being increasingly tackled using software techniques. However, for an inherent nonlinear transducer, a software solution depends upon the proper approach through mathematical modeling of the response curve. An artificial neural network based virtual curve tracer is proposed for modeling, testing and calibration of transducers. The implemented functions are described. Experimental results of the comparative study in modeling the static response characteristic of a strain-gauge pressure transducer using adaptive single-layer linear network and multilayer feed-forward back-propagation network are presented.