Multilayer Perceptron Approach Research Articles

LSTM DSS Automatism and Dataset Optimization for Diabetes Prediction

The paper is focused on the application of Long Short-Term Memory (LSTM) neural network enabling patient health status prediction focusing the attention on diabetes. The proposed topic is an upgrade of a Multi-Layer Perceptron (MLP) algorithm that can be fully embedded into an Enterprise Resource Planning (ERP) platform. The LSTM approach is applied for multi-attribute data processing and it is integrated into an information system based on patient management. To validate the proposed model, we have adopted a typical dataset used in the literature for data mining model testing. The study is focused on the procedure to follow for a correct LSTM data analysis by using artificial records (LSTM-AR-), improving the training dataset stability and test accuracy if compared with traditional MLP and LSTM approaches. The increase of the artificial data is important for all cases where only a few data of the training dataset are available, as for more practical cases. The paper represents a practical application about the LSTM approach into the decision support systems (DSSs) suitable for homecare assistance and for de-hospitalization processes. The paper goal is mainly to provide guidelines for the application of LSTM neural network in type I and II diabetes prediction adopting automatic procedures. A percentage improvement of test set accuracy of 6.5% has been observed by applying the LSTM-AR- approach, comparing results with up-to-date MLP works. The LSTM-AR- neural network can be applied as an alternative approach for all homecare platforms where not enough training sequential dataset is available.

Experimental based multilayer perceptron approach for prediction of evacuated solar collector performance in humid subtropical regions

Solar collectors are efficient in utilising solar thermal energy for heating applications as their efficiency is quite high even in the medium temperature range, which motivated to design a high efficient collector system. In this study, an experimental investigation is carried out by developing a series of evacuated tube solar collectors with U-tube configuration using water as a working fluid. The performance of the collector system is continuously measured throughout the day. A trade-off study is carried out considering all the performance parameters. On the basis of experimental datasets, a multilayer perceptron (MLP) architecture is developed to predict thermal efficiency, useful heat gain and water outlet temperature of the evacuated tube collector as a function of solar irradiation, mass flow rate of water and water inlet temperature. It is demonstrated that the MLP model has an excellent agreement with experimental data as the mean square error is very low (<0.001). Test results showed that the MLP architecture gives a precise prediction of the actual collector performance parameters for different operating conditions. Test results also indicate that MLP model is a robust prediction platform for evaluating the solar collector performance.

Passive sonar target classification using multi-layer perceptron trained by salp swarm algorithm

Due to the variability of the radiated signal of the sonar targets, passive sonar target classification is a challenging problem in the real world application. Adaptive Mel-frequency cepstral coefficients (MFCCs) and multi-layer perceptron (MLP) approaches are proposed, including cepstral features to alleviate dataset's dimension and MLP network to adapt the variability in changing condition. In spite of the capabilities of MLP networks, low classification accuracy, and getting stuck in local minima are the main shortcomings of MLP networks. To overcome these shortcomings, this paper proposes the use of the newly introduced salp swarm algorithm (SSA) for training MLP network. In order to investigate the efficiency of the proposed classifier, four high-dimensional benchmark functions, as well as an experimental passive sonar data set, are employed. The designed classifier is compared to gray wolf optimizer (GWO), biogeography-based optimization (BBO), interior search algorithm (ISA), and group method of data handling (GMDH) in terms of classification accuracy, entrapment in local minima, and convergence speed. The results showed that the proposed classifier is more efficient than the other benchmark algorithms; therefore, the SSA classifies sonar data set as much as 0.9017 percent better than GMDH being the best results among the other classifiers.

A weighted normalized likelihood procedure for empirical land change modeling

A critical foundation for empirical land change modeling is the mapping of transition potentials—quantitative evaluations of the readiness or suitability of land to go through a transition. This paper presents a procedure based on empirically determined normalized likelihoods of transition. It shows that these normalized likelihoods equate to posterior probabilities if case–control sampling is carried out among historical instances of change and persistence. The posterior probabilities can then be aggregated at the pixel level across multiple covariates using linear opinion pooling where the pixel-specific weight for each covariate is determined locally by its ability to distinguish between the alternatives of change or persistence. Thus, covariates with spatially varying diagnostic ability can be productively incorporated. The resulting algorithm is shown to have a skill comparable to that of a multi-layer perceptron approach with the advantage of high efficiency and amenability to distributed processing in a cloud environment.

PM2.5 concentration modeling and mapping in the urban areas

The aim of our study was to compare and evaluate four models by which the ground-level PM2.5 concentrations in the urban area are predicted. These are ordinary kriging (OK), universal kriging (UK), inverse distance weighting (IDW), and multi-layer perceptron (MLP). Root mean square error (RMSE) and standard deviation (SD) were used to select semi-variogram and the transformation type of renormalized data in kriging models. The spherical semi-variogram, Box-Cox with the power parameter equal to one, and first order trend of removal were used to run OK and UK models. In the IDW method, the weighing was conducted according to the distance between points and the measurement station. The trial and error method along with a coefficient of determination (R2) and the lowest root mean square error (RMSE) values were used to obtain the optimum model in MLP approach. Regarding the mapping, the nearest area to main roads showed generally the highest annual average concentrations of PM2.5. Based on the RMSE and R2 of models, the order of fitted models of PM2.5 concentrations in the air from the best to worst was as follows: MLP > OK > UK > IDW.

Probabilistic artificial neural network and E-nose based classification of Rhyzopertha dominica infestation in stored rice grains

The versatility of artificial neural network with back propagation multilayer perceptron approach could entitle an easy and methodical interpretation of results corresponding to multiple metal oxides sensor in an electronic nose. Three algorithms discriminant factorial analysis (DFA), soft independent modeling by class analogy (SIMCA), probabilistic artificial neural network (PANN) with back propagation multilayer perceptron (BPNN) were used for the classification of R. dominica infested rice stored for 225 days via 18 metal oxide sensors in E-nose. The coefficient of correlation for the three approaches were 88 (DFA), 96 (SIMCA), 98.96 (BPNN) %, respectively. The percentage discrimination index was more distinct between 0 and 225 days R. dominica infested rice (98%) than 0–180 days (93%), and 0–135 days (88%). The residual errors of validation and cross validation for SIMCA were 1.04 × 10−3 and 1.26 × 10−3 respectively. Major metal oxide sensors responsible for the production of volatiles were P30/1, T 30/1, PA/2, P30/2, T70/2, P40/1, and P40/2. The overall relative errors during artificial neural network training and testing were 0.092 and 0.286 respectively. The artificial neural network relative error for scale dependents in response to metal oxide sensors for mean, SD, % RSD were 0.033, 0.162, 0.081, respectively. The applicability of E-nose with neural network could help in securing the data analysis time without loss of information and can also work well for noxious odors which might not be able to be categorized by human olfactory.

Contrasting Advantages of Learning With Random Weights and Backpropagation in Non-Volatile Memory Neural Networks

Recently, a Cambrian explosion of a novel, non-volatile memory (NVM) devices known as memristive devices have inspired effort in building hardware neural networks that learn like the brain. Early experimental prototypes built simple perceptrons from nanosynapses, and recently, fully-connected multi-layer perceptron (MLP) learning systems have been realized. However, while backpropagating learning systems pair well with high-precision computer memories and achieve state-of-the-art performances, this typically comes with a massive energy budget. For future Internet of Things/peripheral use cases, system energy footprint will be a major constraint, and emerging NVM devices may fill the gap by sacrificing high bit precision for lower energy. In this paper, we contrast the well-known MLP approach with the extreme learning machine (ELM) or NoProp approach, which uses a large layer of random weights to improve the separability of high-dimensional tasks, and is usually considered inferior in a software context. However, we find that when taking the device non-linearity into account, NoProp manages to equal hardware MLP system in terms of accuracy. While also using a sign-based adaptation of the delta rule for energy-savings, we find that NoProp can learn effectively with four to six ’bits’ of device analog capacity, while MLP requires eight-bit capacity with the same rule. This may allow the requirements for memristive devices to be relaxed in the context of online learning. By comparing the energy footprint of these systems for several candidate nanosynapses and realistic peripherals, we confirm that memristive NoProp systems save energy compared with MLP systems. Lastly, we show that ELM/NoProp systems can achieve better generalization abilities than nanosynaptic MLP systems when paired with pre-processing layers (which do not require backpropagated error). Collectively, these advantages make such systems worthy of consideration in future accelerators or embedded hardware.

Establishing an ANN-Based Risk Model for Ground Subsidence Along Railways

Ground subsidence occurrences have drastically increased in the Seoul area of the Republic of Korea. The structural defects of underground utilities were found to be the primary cause of ground subsidence based on several field investigations. This paper presents a risk model that assesses the probability of occurrence of ground subsidence along railways. In this study, support vector machine (SVM) and multi-layer perceptron (MLP) approaches were successfully employed to develop an artificial neural network (ANN)-based risk model. The risk model, in conjunction with a database composed of underground utilities and geological boring data along urban railway networks, was utilized to develop a hazard map system. A limited field experimental program was conducted for the purpose of verification, resulting in a promising tool to effectively maintain railway networks.

Improvement of Adequate Digoxin Dosage: An Application of Machine Learning Approach.

Digoxin is a high-alert medication because of its narrow therapeutic range and high drug-to-drug interactions (DDIs). Approximately 50% of digoxin toxicity cases are preventable, which motivated us to improve the treatment outcomes of digoxin. The objective of this study is to apply machine learning techniques to predict the appropriateness of initial digoxin dosage. A total of 307 inpatients who had their conditions treated with digoxin between 2004 and 2013 at a medical center in Taiwan were collected in the study. Ten independent variables, including demographic information, laboratory data, and whether the patients had CHF were also noted. A patient with serum digoxin concentration being controlled at 0.5–0.9 ng/mL after his/her initial digoxin dosage was defined as having an appropriate use of digoxin; otherwise, a patient was defined as having an inappropriate use of digoxin. Weka 3.7.3, an open source machine learning software, was adopted to develop prediction models. Six machine learning techniques were considered, including decision tree (C4.5), k-nearest neighbors (kNN), classification and regression tree (CART), randomForest (RF), multilayer perceptron (MLP), and logistic regression (LGR). In the non-DDI group, the area under ROC curve (AUC) of RF (0.912) was excellent, followed by that of MLP (0.813), CART (0.791), and C4.5 (0.784); the remaining classifiers performed poorly. For the DDI group, the AUC of RF (0.892) was the best, followed by CART (0.795), MLP (0.777), and C4.5 (0.774); the other classifiers' performances were less than ideal. The decision tree-based approaches and MLP exhibited markedly superior accuracy performance, regardless of DDI status. Although digoxin is a high-alert medication, its initial dose can be accurately determined by using data mining techniques such as decision tree-based and MLP approaches. Developing a dosage decision support system may serve as a supplementary tool for clinicians and also increase drug safety in clinical practice.

Exploiting behavioral user models for point of interest recommendation in smart museums

ABSTRACTThe Internet of Things (IoT) holds the promise to blend real-world and online behaviors in principled ways, yet we are only beginning to understand how to effectively exploit insights from the online realm into effective applications in smart environments. Such smart environments aim to provide an improved, personalized experience based on the trail of user interactions with smart devices, but how does recommendation in smart environments differ from the usual online recommender systems? And can we exploit similarities to truly blend behavior in both realms to address the fundamental cold-start problem? In this article, we experiment with behavioral user models based on interactions with smart devices in a museum, and investigate the personalized recommendation of what to see after visiting an initial set of Point of Interests (POIs), a key problem in personalizing museum visits or tour guides, and focus on a critical one-shot POI recommendation task—where to go next? We have logged users' onsite physical information interactions during visits in an IoT-augmented museum exhibition at scale. Furthermore, we have collected an even larger set of search logs of the online museum collection. Users in both sets are unconnected, for privacy reasons we do not have shared IDs. We study the similarities between users' online digital and onsite physical information interaction behaviors, and build new behavioral user models based on the information interaction behaviors in (i) the physical exhibition space, (ii) the online collection, or (iii) both. Specifically, we propose a deep neural multilayer perceptron (MLP) based on explicitly given users' contextual information, and set-based extracted features using users' physical information interaction behaviors and similar users' digital information interaction behaviors. Our experimental results indicate that the proposed behavioral user modeling approach, using both physical and online user information interaction behaviors, improves the onsite POI recommendation baselines' performances on all evaluation metrics. Our proposed MLP approach achieves 83% precision at rank 1 on the critical one-shot POI recommendation problem, realizing the high accuracy needed for fruitful deployment in practical situations. Furthermore, the MLP model is less sensitive to amount of real-world interactions in terms of the seen POIs set-size, by backing of to the online data, hence helps address the cold start problem in recommendation. Our general conclusion is that it is possible to fruitfully combine information interactions in the online and physical world for effective recommendation in smart environments.

T-wave end detection using neural networks and Support Vector Machines

Background and objectiveIn this paper we propose a new approach for detecting the end of the T-wave in the electrocardiogram (ECG) using Neural Networks and Support Vector Machines. MethodsBoth, Multilayer Perceptron (MLP) neural networks and Fixed-Size Least-Squares Support Vector Machines (FS-LSSVM) were used as regression algorithms to determine the end of the T-wave. Different strategies for selecting the training set such as random selection, k-means, robust clustering and maximum quadratic (Rényi) entropy were evaluated. Individual parameters were tuned for each method during training and the results are given for the evaluation set. A comparison between MLP and FS-LSSVM approaches was performed. Finally, a fair comparison of the FS-LSSVM method with other state-of-the-art algorithms for detecting the end of the T-wave was included. ResultsThe experimental results show that FS-LSSVM approaches are more suitable as regression algorithms than MLP neural networks. Despite the small training sets used, the FS-LSSVM methods outperformed the state-of-the-art techniques. ConclusionFS-LSSVM can be successfully used as a T-wave end detection algorithm in ECG even with small training set sizes.

A Multi-layer Perceptron Approach to Automatically Detect Tissue via NIR Multispectral Imaging

We present a novel pixel-level spectra based multi-layer perceptron(MLP) to discriminate regions of biomedical multispectral imagingdata into two categories: tissue and non-tissue. The spectra usedfor this study are 740nm, 780nm, 850nm, and 945nm as thesewavelengths are on either side of the isosbestic point for oxyhemoglobinand deoxyhemoglobin; absorbers that are common in allhealthy tissues. An MLP is trained using multispectral data from12 human subjects and 12 non-tissue objects. The MLP is testedon three multispectral challenge image sets, from which the accuracy,sensitivity, and specificity of the model yield results of 91.3%(+/-0.2%), 98.1% (+/-0.3%), and 88.5% (+/- 0.3%) respectively.

An Optimum Neural Network for Evolutionary Aerodynamic Shape Design

Two new techniques are proposed to enhance the estimation abilities of the conventional neural network (NN) method for its application to the fitness function estimation of aerodynamic shape optimization with the genetic algorithm (GA). The first technique is pre-processing the training data in order to increase the training accuracy of the multi-layer perceptron (MLP) approach. The second technique is a new structure for the network to improve its quality through a modified growing and pruning method. Using the proposed techniques, one can obtain the best estimations from the NN with less computational time. The new methods are applied for optimum design of a transonic airfoil and the results are compared with those obtained from the accurate Computational Fluid Dynamics (CFD) fitness evaluator and also with the conventional MLP NN approach. The numerical experiments show that using the new method can reduce the computational time significantly while achieving the improved accuracy.

Artificial neural network for suppression of banding artifacts in balanced steady-state free precession MRI

The balanced steady-state free precession (bSSFP) MR sequence is frequently used in clinics, but is sensitive to off-resonance effects, which can cause banding artifacts. Often multiple bSSFP datasets are acquired at different phase cycling (PC) angles and then combined in a special way for banding artifact suppression. Many strategies of combining the datasets have been suggested for banding artifact suppression, but there are still limitations in their performance, especially when the number of phase-cycled bSSFP datasets is small. The purpose of this study is to develop a learning-based model to combine the multiple phase-cycled bSSFP datasets for better banding artifact suppression. Multilayer perceptron (MLP) is a feedforward artificial neural network consisting of three layers of input, hidden, and output layers. MLP models were trained by input bSSFP datasets acquired from human brain and knee at 3T, which were separately performed for two and four PC angles. Banding-free bSSFP images were generated by maximum-intensity projection (MIP) of 8 or 12 phase-cycled datasets and were used as targets for training the output layer. The trained MLP models were applied to another brain and knee datasets acquired with different scan parameters and also to multiple phase-cycled bSSFP functional MRI datasets acquired on rat brain at 9.4T, in comparison with the conventional MIP method. Simulations were also performed to validate the MLP approach. Both the simulations and human experiments demonstrated that MLP suppressed banding artifacts significantly, superior to MIP in both banding artifact suppression and SNR efficiency. MLP demonstrated superior performance over MIP for the 9.4T fMRI data as well, which was not used for training the models, while visually preserving the fMRI maps very well. Artificial neural network is a promising technique for combining multiple phase-cycled bSSFP datasets for banding artifact suppression.

Prediction of Fe-Co-Mn/MgO Catalytic Activity in Fischer-Tropsch Synthesis Using Nu-support Vector Regression

Support vector regression (SVR) is a learning method based on the support vector machine (SVM) that can be used for curve fitting and function estimation. In this paper, the ability of the nu-SVR to predict the catalytic activity of the Fischer-Tropsch (FT) reaction is evaluated and the result is compared with two other prediction techniques including: multilayer perceptron (MLP) and subtractive clustering-adaptive neuro-fuzzy inference system (SUB-ANFIS). The Fischer-Tropsch synthesis (FTS) was studied in a fixed bed micro-reactor under different operating conditions. An extensive experimental data set of MgO supported Fe-Co-Mn catalyst was used to predict the FTS. The input variables of three aforesaid models were: reactor temperature, H2/CO ratio and total pressure, while the CO conversion (catalytic activity) was used as an output variable. Finally, the achieved results from these approaches were compared. The results reveal that thenu-SVR model has more accurate (MSE = 0.0014) than the MLP (MSE = 0.0097) and ANFIS (MSE = 0.0043) approaches.

Multilayer Perceptron approach for estimating 5-min and hourly horizontal global irradiation from exogenous meteorological data in locations without solar measurements

Only one thousand stations around the world measures solar radiation sometimes with a poor quality. The objective of this paper is to show if solar irradiations at short time scale, hourly and 5-min, (very under-studied time-step) can be estimated from more available and cheaper data using Artificial Neural Networks. 7 meteorological and 3 calculated parameters are used as inputs; 1023 inputs combinations are possible for each time-step; the best inputs combinations are pursued. A variable selection method based on Pearson's coefficient is firstly used between inputs and between output and inputs; some inputs are redundant (particularly calculated ones) and/or with a weak link with solar radiation (as wind speed and direction), sunshine duration is strongly correlated with solar irradiation. The models have a good adequacy mainly with sunshine duration in the input set. For hourly data, the performances of the 6 and 10 inputs model are nRMSE = 13.90% (nMAE = 13.28%, R2 = 0.979) and nRMSE = 13.33% (nMAE = 12.72%, R2 = 0.9812); without sunshine duration, the model nRMSE (with 5 inputs) falls to 28.27%. For 5-min data, the 6 and 10 inputs models have a nRMSE equal to 19.35% and 18.65% which is very good for such a time-step. A comparison with literature highlighted the quality of these models.

Hourly global solar irradiation forecasting for New Zealand

With the growing use of solar thermal energy systems and small scale photovoltaic power generation by domestic users, there is increasing need to develop intelligent controllers that allow these users to efficiently manage the energy generated by these systems. Ideally these intelligent controllers will be able to forecast the availability and magnitude of the solar resource to plan in advance for periods when the solar irradiance magnitude is small or unavailable. In addition, the method used to provide this forecast needs to be adaptable to a range of timescales and locations. With this in mind, this study examined the possibility of providing a 24-h ahead forecast of hourly global solar irradiation in New Zealand using several approaches but with particular reference to nonlinear autoregressive recurrent neural networks with exogenous inputs (NARX).Hourly time series data for nine historic weather variables recorded over a three year period was used to train and test the forecasting methods for New Zealand’s largest city, Auckland. Results from forecasts based on the NARX were compared with an artificial neural network (ANN) based Multilayer Perceptron (MLP) method, a statistical approach using auto regressive moving average (ARMA) and a reference persistence approach. Predicted values of hourly global solar irradiation were compared with the measured values, and it was found that the root mean squared error (RMSE) was 0.243MJ/m2 for the NARX method as compared to 0.484MJ/m2, 0.315MJ/m2 and 0.514MJ/m2 for the MLP, ARMA and persistence approaches respectively. Subsequently the NARX approach was used to forecast global solar irradiation for other major cities across New Zealand. The results demonstrate the ability of the NARX approach to forecast irradiation values at a later time and across a number of different locations. As such it is foreseeable that such an approach could serve as the basis of a forecasting system in future intelligent controllers.

Artificial chemical reaction optimization of neural networks for efficient prediction of stock market indices

Abstract The underlying system models of time series prediction are complex and not known a priori, hence, accurate and unbiased estimation cannot be always achieved using well known linear techniques. The estimation process requires more advanced prediction algorithms, such as multilayer perceptrons (MLPs). This paper presents an artificial chemical reaction neural network (ACRNN), which uses artificial chemical reaction optimization (ACRO) to train the MLP models for forecasting the stock market indices. The underlying motivation for using ACRO is the ability to overcome the issues of convergence, parameter setting and overfitting and to accurately forecast financial time series data even when the underlying system processes are typically nonlinear. Historical data of seven different stock indices have been collected for 15 years to test the performance of the ACRNN approach. After extensive experimentation, it is observed that the ACRNN technique demonstrates significant improvements in prediction accuracy over the MLP approach.

Mapping regional forest fire probability using artificial neural network model in a Mediterranean forest ecosystem

ABSTRACTForest fires are one of the most important factors in environmental risk assessment and it is the main cause of forest destruction in the Mediterranean region. Forestlands have a number of known benefits such as decreasing soil erosion, containing wild life habitats, etc. Additionally, forests are also important player in carbon cycle and decreasing the climate change impacts. This paper discusses forest fire probability mapping of a Mediterranean forestland using a multiple data assessment technique. An artificial neural network (ANN) method was used to map forest fire probability in Upper Seyhan Basin (USB) in Turkey. Multi-layer perceptron (MLP) approach based on back propagation algorithm was applied in respect to physical, anthropogenic, climate and fire occurrence datasets. Result was validated using relative operating characteristic (ROC) analysis. Coefficient of accuracy of the MLP was 0.83. Landscape features input to the model were assessed statistically to identify the most descriptive factors on forest fire probability mapping using the Pearson correlation coefficient. Landscape features like elevation (R = −0.43), tree cover (R = 0.93) and temperature (R = 0.42) were strongly correlated with forest fire probability in the USB region.

A Multi-Layer Perceptron Approach for Customer Churn Prediction

A Multi-Layer Perceptron Approach for Customer Churn Prediction