MLP-based Model Research Articles

Comparison and optimization of deep learning-based radiosensitivity prediction models using gene expression profiling in National Cancer Institute-60 cancer cell line

BackgroundIn this study, various types of deep-learning models for predicting in vitro radiosensitivity from gene-expression profiling were compared. MethodsThe clonogenic surviving fractions at 2 Gy from previous publications and microarray gene-expression data from the National Cancer Institute-60 cell lines were used to measure the radiosensitivity. Seven different prediction models including three distinct multi-layered perceptrons (MLP), four different convolutional neural networks (CNN) were compared. Folded cross-validation was applied to train and evaluate model performance. The criteria for correct prediction were absolute error < 0.02 or relative error < 10%. The models were compared in terms of prediction accuracy, training time per epoch, training fluctuations, and required calculation resources. ResultsThe strength of MLP-based models was their fast initial convergence and short training time per epoch. They represented significantly different prediction accuracy depending on the model configuration. The CNN-based models showed relatively high prediction accuracy, low training fluctuations, and a relatively small increase in the memory requirement as the model deepens. ConclusionOur findings suggest that a CNN-based model with moderate depth would be appropriate when the prediction accuracy is important, and a shallow MLP-based model can be recommended when either the training resources or time are limited.

MLP-Based Model for Estimation of Methane Seam Pressure

One of the principal indicators of the methane hazard in coal mines is gas pressure. This parameter directly affects the methane content in the seam as well as the rate of its release resulting from mining operations. Because of limitations in the existing methods for methane seam pressure measuring, primarily technical difficulties associated with direct measurement and the time-consuming nature of indirect measurement, this parameter is often disregarded in the coal and gas outburst forecasts. To overcome the above-mentioned difficulties, an attempt was made to estimate the methane seam pressure with the use of artificial neural networks. Two MLP-based models were developed to estimate the average and maximum methane seam pressure values, respectively. The analyses demonstrated high correlation between the values indicated by the neural models and the reference values determined on the basis of sorption isotherms. According to the adopted fit criterion, the prediction errors for the best fit were 2.59% and 3.04% for the average and maximum seam pressure values, respectively. The obtained determination coefficients (exceeding the value of 0.99) confirmed the very good predictive abilities of the models. These results imply a great potential for practical application of the proposed method.

Forecasting COVID-19 recovered cases with Artificial Neural Networks to enable designing an effective blood supply chain

This study introduces a forecasting model to help design an effective blood supply chain mechanism for tackling the COVID-19 pandemic. In doing so, first, the number of people recovered from COVID-19 is forecasted using the Artificial Neural Networks (ANNs) to determine potential donors for convalescent (immune) plasma (CIP) treatment of COVID-19. This is performed explicitly to show the applicability of ANNs in forecasting the daily number of patients recovered from COVID-19. Second, the ANNs-based approach is further applied to the data from Italy to confirm its robustness in other geographical contexts. Finally, to evaluate its forecasting accuracy, the proposed Multi-Layer Perceptron (MLP) approach is compared with other traditional models, including Autoregressive Integrated Moving Average (ARIMA), Long Short-term Memory (LSTM), and Nonlinear Autoregressive Network with Exogenous Inputs (NARX). Compared to the ARIMA, LSTM, and NARX, the MLP-based model is found to perform better in forecasting the number of people recovered from COVID-19. Overall, the findings suggest that the proposed model is robust and can be widely applied in other parts of the world in forecasting the patients recovered from COVID-19.

A comprehensive comparison of recent developed meta-heuristic algorithms for streamflow time series forecasting problem

Hydrological models play a crucial role in water planning and decision making. Machine Learning-based models showed several drawbacks for frequent high and a wide range of streamflow records. These models also experience problems during the training process such as over-fitting or trapping in searching for global optima To overcome these limitations, the current study attempts to hybridize the recently developed physics-inspired metaheuristic algorithms (MHAs) such as Equilibrium Optimization (EO), Henry Gases Solubility Optimization (HGSO), and Nuclear Reaction Optimization(NRO) with Multi-layer Perceptron (MLP). These models’ accuracy will be inspected to solve the streamflow forecasting problem where the streamflow dataset was collected through 130 years from a station located on the High Aswan Dam (HAD). The performance of proposed models then will be compared with two traditional neural network models(MLP and RNN), and nine well-known hybrid MLP-based models belong to the different branches of the metaheuristic field (evolutionary group, swarm group, and physics group). The internal parameters of the proposed models will be initialized and optimized. Different performance metrics will be used to examine the performance of the proposed models. The stability of the proposed models and the convergence speed will be evaluated. Finally, ranking these models based on different performance evaluations will be carried out. The results show that the models in the group of Physics-MLP are more reliable in capturing the streamflow patterns, followed by the Swarm-MLP group and then by the Evolutionary-MLP group. Finally, among the all employed methods, the NRO has the best accuracy with the lowest RMSE(2.35), MAE(1.356), MAPE(16.747), and the highest WI(0.957), R(0.924), and confidence in forecasting the streamflow of Aswan High Dam. It can be concluded that augmenting the NRO algorithm with MLP can be a reliable tool in forecasting the monthly streamflow with a high level of precision, speed convergence, and high constancy level.

Seismic Ground Response Analysis Based on Multilayer Perceptron and Convolution Neural Networks

The importance of establishing a disaster prevention plan considering seismic performance is being highlighted to reduce damage to structures caused by earthquakes. Earthquake waves propagate from the bedrock to the ground surface through the soil. During the transmission process, they are amplified in a specific frequency range, and the degree of amplification depends mainly on the characteristics of the ground. Therefore, a seismic response analysis process is essential for enhancing the reliability of the seismic design. We propose a model for predicting seismic waves on the surface from seismic waves measured on the bedrock based on Multilayer Perceptron (MLP) and Convolutional Neural Networks (CNN) and validate the applicability of the proposed model with Spectral Acceleration (SA). Both the proposed models based on MLP and CNN successfully predicted the seismic response of the surface. The CNN-based model performed better than the MLP-based model, with a 10% smaller average error. We plan to implement the physical properties of the ground, such as shear wave velocity, to create a more versatile model in the future.

On the Anatomy of Predictive Models for Accelerating GPU Convolution Kernels and Beyond

Efficient HPC libraries often expose multiple tunable parameters, algorithmic implementations, or a combination of them, to provide optimized routines. The optimal parameters and algorithmic choices may depend on input properties such as the shapes of the matrices involved in the operation. Traditionally, these parameters are manually tuned or set by auto-tuners. In emerging applications such as deep learning, this approach is not effective across the wide range of inputs and architectures used in practice. In this work, we analyze different machine learning techniques and predictive models to accelerate the convolution operator and GEMM. Moreover, we address the problem of dataset generation, and we study the performance, accuracy, and generalization ability of the models. Our insights allow us to improve the performance of computationally expensive deep learning primitives on high-end GPUs as well as low-power embedded GPU architectures on three different libraries. Experimental results show significant improvement in the target applications from 50% up to 300% compared to auto-tuned and high-optimized vendor-based heuristics by using simple decision tree- and MLP-based models.

Artificial neural network-based prediction of prolonged length of stay and need for post-acute care in acute coronary syndrome patients undergoing percutaneous coronary intervention.

Prolonged length of stay (LOS) and post-acute care after percutaneous coronary intervention (PCI) is common and costly. Risk models for predicting prolonged LOS and post-acute care have limited accuracy. Our goal was to develop and validate models using artificial neural networks (ANN) to predict prolonged LOS>7days and need for post-acute care after PCI. We defined prolonged LOS as ≥7days and post-acute care as patients discharged to: extended care, transitional care unit, rehabilitation, other acute care hospital, nursing home or hospice care. Data from 22675 patients who presented with ACS and underwent PCI was shuffled and split into a derivation set (75% of dataset) and a validation dataset (25% of dataset). Calibration plots were used to examine the overall predictive performance of the MLP by plotting observed and expected risk deciles and fitting a lowess smoother to the data. Classification accuracy was assessed by a receiver-operating characteristic (ROC) and area under the ROC curve (AUC). Our MLP-based model predicted prolonged LOS with an accuracy of 90.87% and 88.36% in training and test sets, respectively. The post-acute care model had an accuracy of 90.22% and 86.31% in training and test sets, respectively. This accuracy was achieved with quick convergence. Predicted probabilities from the MLP models showed good (prolonged LOS) to excellent calibration (post-acute care). Our ANN-based models accurately predicted LOS and need for post-acute care. Larger studies for replicability and longitudinal studies for evidence of impact are needed to establish these models in current PCI practice.

Forecasting global coal consumption: An artificial neural network approach

Abstract In the 21st century, energy has become an integral part of our society and of global economic development. Although the world has experienced tremendous technological advancements, fossil fuels (including coal, natural gas, and oil) continue to be the world’s primary energy source. At the current production level, it has been estimated that coal reserves (economically recoverable) would last approximately 130 years (with the biggest reserves found in the USA, Russia, China, and India). The intricate relationship between economic growth, demographics and energy consumption (particularly in countries with coal intensive industries and heavy reliance on fossil fuels), along with the elevated amounts of greenhouse gases in the atmosphere, have raised serious concerns within the scientific community about the future of coal. Thus, various studies have focused on the development and application of forecasting methods to predict the economic prospects of coal, future levels of reserves, production, consumption, and its environmental impact. With this scope in mind, the goal of this article is to contribute to the scarce literature on global coal consumption forecasting with the aid of an artificial neural network method. This paper proposes a Multilayer Perceptron neural network (MLP) for the prediction of global coal consumption for the years 2020-2030. The MLP-based model is trained with historical data sets gathered from financial institutions, global energy authorities, and energy statistic agencies, covering the years 1970 through 2016. The results of this study show a deceleration in global coal consumption for the years 2020 (3 932 Mtoe), 2025 (4 069 Mtoe) and 2030 (4 182 Mtoe).

Learning from Few Samples with Memory Network

Neural networks (NN) have achieved great successes in pattern recognition and machine learning. However, the success of a NN usually relies on the provision of a sufficiently large number of data samples as training data. When fed with a limited data set, a NN’s performance may be degraded significantly. In this paper, a novel NN structure is proposed called a memory network. It is inspired by the cognitive mechanism of human beings, which can learn effectively, even from limited data. Taking advantage of the memory from previous samples, the new model achieves a remarkable improvement in performance when trained using limited data. The memory network is demonstrated here using the multi-layer perceptron (MLP) as a base model. However, it would be straightforward to extend the idea to other neural networks, e.g., convolutional neural networks (CNN). In this paper, the memory network structure is detailed, the training algorithm is presented, and a series of experiments are conducted to validate the proposed framework. Experimental results show that the proposed model outperforms traditional MLP-based models as well as other competitive algorithms in response to two real benchmark data sets.

Implementation of a hybrid MLP-FFA model for water level prediction of Lake Egirdir, Turkey

The predictive ability of a hybrid model integrating the Firefly Algorithm (FFA), as a heuristic optimization tool with the Multilayer Perceptron (MLP-FFA) algorithm for the prediction of water level in Lake Egirdir, Turkey, is investigated. The accuracy of the hybrid MLP-FFA model is then evaluated against the standalone MLP-based model developed with the Levenberg–Marquadt optimization scheme applied for in the backpropagation-based learning process. To develop and investigate the veracity of the proposed hybrid MLP-FFA model, monthly time scale water level data for 56 years (1961–2016) are applied to train and test the hybrid model. The input combinations of the standalone and the hybrid predictive models are determined in accordance with the Average Mutual Information computed from the historical water level (training) data; generating four statistically significant lagged combinations of historical data to be adopted for the 1-month forecasting of lake water level. The proposed hybrid MLP-FFA model is evaluated with statistical score metrics: Nash–Sutcliffe efficiency, root mean square and mean absolute error, Wilmott’s Index and Taylor diagram developed in the testing phase. The analysis of the results showed that the hybrid MLP–FFA4 model (where 4 months of lagged combinations of lake water level data are utilized) performed more accurately than the standalone MLP4 model. For the fully optimized hybrid (MLP-FFA4) model evaluated in the testing phase, the Willmott’s Index was approximately 0.999 relative to 0.988 (MLP 4) and the root mean square error was approximately 0.029 m and compared to 0.102 m. Moreover, the inter-comparison of the forecasted and the observed data with various other performance metrics (including the Taylor diagram) verified the robustness of the proposed hybrid MLP-FFA4 model over the standalone MLP4 model applied in the problem of forecasting lake water level prediction in the current semi-arid region in Turkey.

Uncertainty assessment of the multilayer perceptron (MLP) neural network model with implementation of the novel hybrid MLP-FFA method for prediction of biochemical oxygen demand and dissolved oxygen: a case study of Langat River

Accurate prediction of the chemical constituents in major river systems is a necessary task for water quality management, aquatic life well-being and the overall healthcare planning of river systems. In this study, the capability of a newly proposed hybrid forecasting model based on the firefly algorithm (FFA) as a metaheuristic optimizer, integrated with the multilayer perceptron (MLP-FFA), is investigated for the prediction of monthly water quality in Langat River basin, Malaysia. The predictive ability of the MLP-FFA model is assessed against the MLP-based model. To validate the proposed MLP-FFA model, monthly water quality data over a 10-year duration (2001–2010) for two different hydrological stations (1L04 and 1L05) provided by the Irrigation and Drainage Ministry of Malaysia are used to predict the biochemical oxygen demand (BOD) and dissolved oxygen (DO). The input variables are the chemical oxygen demand (COD), total phosphate (PO4), total solids, potassium (K), sodium (Na), chloride (Cl), electrical conductivity (EC), pH and ammonia nitrogen (NH4-N). The proposed hybrid model is then evaluated in accordance with statistical metrics such as the correlation coefficient (r), root-mean-square error, % root-mean-square error and Willmott’s index of agreement. Analysis of the results shows that MLP-FFA outperforms the equivalent MLP model. Also, in this research, the uncertainty of a MLP neural network model is analyzed in relation to the predictive ability of the MLP model. To assess the uncertainties within the MLP model, the percentage of observed data bracketed by 95 percent predicted uncertainties (95PPU) and the band width of 95 percent confidence intervals (d-factors) are selected. The effect of input variables on BOD and DO prediction is also investigated through sensitivity analysis. The obtained values bracketed by 95PPU show about 77.7%, 72.2% of data for BOD and 72.2%, 91.6% of data for DO related to the 1L04 and 1L05 stations, respectively. The d-factors have a value of 1.648, 2.269 for BOD and 1.892, 3.480 for DO related to the 1L04 and 1L05 stations, respectively. Based on the values in both stations for the 95PPU and d-factor, it is concluded that the neural network model has an acceptably low degree of uncertainty applied for BOD and DO simulations. The findings of this study can have important implications for error assessment in artificial intelligence-based predictive models applied for water resources management and the assessment of the overall health in major river systems.

Recognition of sketching from surface electromyography

The main objective of this study is to recognize sketching precisely and effectively in human–computer interaction. A surface electromyography (sEMG)-based sketching recognition method is proposed. We conducted an experiment in which we recorded the sEMG signals from the forearm muscles of two participants who were instructed to sketch seven basic one-stroke shapes. Subsequently, seven features of the sEMG time domain were extracted. After reducing data dimensionality using principal component analysis, these features were used as input vectors to a sketching recognition model based on support vector machines (SVMs). The performance of this model was compared to two other recognition models based on multilayer perceptron (MLP) neural networks and self-organization feature map (SOFM) neural networks. The average recognition rates for the seven basic one-stroke shapes of two participants achieved by the SVM-based and MLP-based models were both 98.5% in the test set, which were slightly superior to the performance of the SOFM classifier. Our results demonstrate the feasibility of converting forearm sEMG signals into sketching patterns.

Sun-tracking imaging system for intra-hour DNI forecasts

A Sun-tracking imaging system is implemented for minimizing circumsolar image distortion for improved short-term solar irradiance forecasts. This sky-imaging system consists of a fisheye digital camera mounted on an automatic solar tracker that follows the diurnal pattern of the Sun. The Sun is located at the geometric center of the sky images where the fisheye distortion is minimized. Images from this new system provide more information about the circumsolar sky cover, which provides critical information for intra-hour solar forecasts, particularly for direct normal irradiance. An automatic masking algorithm has been developed to separate the sky area from ground obstacles and the image edges for each image that is collected. Then numerical image features are extracted from the segmented sky area and are used as exogenous inputs to MultiLayer Perceptron (MLP) models for direct normal irradiance forecasts. Sixty-seven days of irradiance and image measurements are used to train, optimize, and assess the MLP-based forecast models for solar irradiance. The results show that the MLP forecasts based on the newly proposed sky-imaging system significantly outperform the reference models in terms of statistical metrics and forecast skill, particularly for shorter horizons, achieving forecast skills 18%–50% higher than the skills of a reference MLP-based model that is based on a zenith-pointed, stationary sky-imaging system.

Artificial neural network modeling of hydrogen-rich syngas production from methane dry reforming over novel Ni/CaFe2O4 catalysts

In this study, the application of artificial neural networks (ANN) for the modeling of hydrogen-rich syngas produced from methane dry reforming over Ni/CaFe2O4 catalysts was investigated. Multi-layer perceptron (MLP) and radial basis function (RBF) neural network architectures were employed for the modeling of the experimental data obtained from methane dry reforming over novel Ni/CaFe2O3 catalysts. The Ni/CaFe2O3 catalysts were synthesized and characterized by XRD, SEM, EDX and FTIR. The as-synthesized Ni/CaFe2O3 catalysts were tested in a continuous flow fixed bed stainless steel reactor for the production of hydrogen-rich syngas via methane dry reforming. The inputs to the ANN–MLP and ANN–RBF-based models were the catalyst metal loadings (5–15wt %), feed ratio (0.4–1.0) and the reaction temperature (700–800 °C). The two models were statistically discriminated in order to measure their predictive capability for the hydrogen-rich syngas production. Coefficient of determination (R2) values of 0.9726, 0.8597, 0.9638 and 0.9394 obtained from the prediction of H2 yield, CO yield, CH4 conversion and CO2 conversion respectively using ANN–MLP-based model were higher compared to R2 values of 0.9218, 0.7759, 0.8307 and 0.7425 obtained for the prediction of H2 yield, CO yield, CH4 conversion and CO conversion respectively using ANN–RBF-based model. The statistical results showed that the ANN–MLP-based model performed better than ANN–RBF model for the prediction of hydrogen-rich syngas from methane dry reforming over the Ni/CaFe2O4 catalysts. Further t-test performed based on the target outputs from the ANN–MLP and ANN–RBF network shows that the models were statistically significant.

Prediction of Higher Heating Value of Solid Biomass Fuels Using Artificial Intelligence Formalisms

The higher heating value (HHV) is an important property defining the energy content of biomass fuels. A number of proximate and/or ultimate analysis based predominantly linear correlations have been proposed for predicting the HHV of biomass fuels. A scrutiny of the relationships between the constituents of the proximate and ultimate analyses and the corresponding HHVs suggests that all relationships are not linear and thus nonlinear models may be more appropriate. Accordingly, a novel artificial intelligence (AI) formalism, namely genetic programming (GP) has been employed for the first time for developing two biomass HHV prediction models, respectively using the constituents of the proximate and ultimate analyses as the model inputs. The prediction and generalization performance of these models was compared rigorously with the corresponding multilayer perceptron (MLP) neural network based as also currently available high-performing linear and nonlinear HHV models. This comparison reveals that the HHV prediction performance of the GP and MLP models is consistently better than that of their existing linear and/or nonlinear counterparts. Specifically, the GP- and MLP-based models exhibit an excellent overall prediction accuracy and generalization performance with high (>0.95) magnitudes of the coefficient of correlation and low (<4.5 %) magnitudes of mean absolute percentage error in respect of the experimental and model-predicted HHVs. It is also found that the proximate analysis-based GP model has outperformed all the existing high-performing linear biomass HHV prediction models. In the case of ultimate analysis-based HHV models, the MLP model has exhibited best prediction accuracy and generalization performance when compared with the existing linear and nonlinear models. The AI-based models introduced in this paper due to their excellent performance have the potential to replace the existing biomass HHV prediction models.

Improving the management of microfinance institutions by using credit scoring models based on Statistical Learning techniques

A wide range of supervised classification algorithms have been successfully applied for credit scoring in non-microfinance environments according to recent literature. However, credit scoring in the microfinance industry is a relatively recent application, and current research is based, to the best of our knowledge, on classical statistical methods. This lack is surprising since the implementation of credit scoring based on supervised classification algorithms should contribute towards the efficiency of microfinance institutions, thereby improving their competitiveness in an increasingly constrained environment. This paper explores an extensive list of Statistical Learning techniques as microfinance credit scoring tools from an empirical viewpoint. A data set of microcredits belonging to a Peruvian Microfinance Institution is considered, and the following models are applied to decide between default and non-default credits: linear and quadratic discriminant analysis, logistic regression, multilayer perceptron, support vector machines, classification trees, and ensemble methods based on bagging and boosting algorithm. The obtained results suggest the use of a multilayer perceptron trained in the R statistical system with a second order algorithm. Moreover, our findings show that, with the implementation of this MLP-based model, the MFIś misclassification costs could be reduced to 13.7% with respect to the application of other classic models.

Nonlinear separation of show-through image mixtures using a physical model trained with ICA

Often, when we scan a document, the image from the back page shows through, due to partial transparency of the paper, giving rise to a mixture of two images. We address the problem of separating these images through the use of a physical model of the mixture process. The model is nonlinear but invertible, and we use the inverse model to perform the separation. The model is trained through the MISEP technique of nonlinear ICA. Bounded independent sources are proved to be separable through this method, apart from offset, scale and permutation indeterminacies. We compare our results with those obtained with other approaches and with different separation models that were trained with MISEP. For the latter case we test a bilinear model and MLP-based models, using both symmetry-based regularization and the more recently proposed minimal nonlinear distortion regularization. Quantitative quality measures show that the approach that we propose is superior to the other methodologies.

On building local models for inverse system identification with vector quantization algorithms

In this paper we provide a comprehensive performance evaluation of vector quantization (VQ) algorithms as building blocks for designing local models for inverse system identification. We describe how VQ algorithms can be used for learning compact representations of the task of interest from available input–output time series data and how this representation can be used to build local maps that approximates the global inverse model of the system. The performances of the resulting local models are compared to the standard global (multilayer perceptron) MLP-based model in the task of inverse modeling of four well-known single input–single output (SISO) systems. The obtained results show that VQ-based local models perform better than the MLP in all the studied tasks.

Flexural strength of creep feed ground ceramics: general pattern, ductile-brittle transition and mlp modeling

Three ceramic materials including two aluminas and one silicon nitride were ground under various creep feed conditions. Flexural strengths of the ground specimens were measured by four-point bending. The experimental data indicates a general flexural strength pattern consisted of three regimes as a function of mean unit load and grit depth of cut (or mean chip cross-sectional area). Defining ductile-brittle transition as the point where strength starts to degrade, it is found that the transition occurs at higher mean unit load, grit depth of cut, or mean chip cross-sectional area for the lower strength material. The percentage of fractured area is found not to be a good indicator of strength for alumina materials. The multi-layer perceptron (MLP) neural network is proposed to establish the strength model. The general form of the MLP-based strength models as well as the fitted parameters are given. The implications of the study are discussed with respect to past studies.

An MLP-based model for identifying qEEG in depression

Manual differentiation of electroencephalography (EEG) paper recordings in cases of depression is not very helpful. So, a Multilayer Perceptron (MLP) has been used to differentiate the EEG power density spectra (qEEG) in the wakeful state from animals (control, exercised and depressed). The qEEG ranging from 1 to 30 Hz, at 1 Hz increments (30 input features) and also as slow, medium and fast activity (represented by three ranges of frequencies at the input) were used. After training with depressed and control qEEG only, the MLP has been found to distinguish successfully between the normal and the depressed rats in more than 80% of the cases, identifying, in the process, most of the exercised groups' EEG as normal. The reduction in the dimension of input features from 30 individual frequencies to 3 frequency bands has produced similar results. The rules generated for making such distinctions have been found to be similar to the clinical views.