Multilayer Perceptron Learning Research Articles

Using multilayer perceptron and similarity-weighted machine learning algorithms to reconstruct the past: A case study of the agricultural expansion on grasslands in the Uruguayan savannas.

Changes in land use and land cover (LULC) have significant implications for biodiversity, ecosystem functioning, and deforestation. Modeling LULC changes is crucial to understanding anthropogenic impacts on environmental conservation and ecosystem services. Although previous studies have focused on predicting future changes, there is a growing need to determine past scenarios using new assessment tools. This study proposes a methodology for LULC past scenario generation based on transition analysis. Aiming to hindcast LULC scenario in 1970 based on the transition analysis of the past 35 years (from 1985 to 2020), two machine learning algorithms, multilayer perceptron (MLP) and similarity weighted (SimWeight), were employed to determine the driver variables most related to conversions in LULC and to simulate the past. The study focused on the Aristida spp. grasslands in the Uruguayan savannas, where native grasslands have been extensively converted to agricultural areas. Land use and land cover data from the MapBiomas project were integrated with spatial variables such as altimetry, slope, pedology, and linear distances from rivers, roads, urban areas, agriculture, forest, forestry, and native grasslands. The accuracy of the predicted maps was assessed through stratified random sampling of reference images from the Multispectral Scanner (MSS) sensor. The results demonstrate a reduction of approximately 659 934 ha of native grasslands in the study area between 1985 and 2020, directly proportional to the increase in cultivable areas. The MLP algorithm exhibited moderate performance, with notable errors in classifying agriculture and grassland areas. In contrast, the SimWeight algorithm displayed better accuracy, particularly in distinguishing grassland and agriculture classes. The modeled map using SimWeight accurately represented the transitions between grassland and agriculture with a high level of agreement. By modeling the 1970s scenario using the SimWeight model, it was estimated that the Aristida spp. grasslands experienced a substantial reduction in grassland coverage, ranging from 9982.31 to 10 022.32 km2 between 1970 and 2020. This represents a range of 60.8%-61.07% of the total grassland area in 1970. These findings provide valuable insights into the driving factors behind land use change in the Aristida spp. grasslands and offer useful information for land management, conservation, and sustainable development in the region. The study's main contribution lies in the hindcasting of past LULC scenarios, utilizing a tool used primarily for forecasting future scenarios. Integr Environ Assess Manag 2024;20:1140-1155. © 2023 SETAC.

Comparison of Artificial Intelligence Techniques for The UK Air Passenger Short-Term Demand Forecasting: A Destination Insight Study

Web search queries become essential drivers to forecast air passenger demand for operational benefits. Scholars and marketing experts. Forecasting passenger demand is one of the most important marketing problems that experts frequently encounter, but there are very few studies in the literature using search queries. The main novelty of this study is to show that Destination Insight (DI) can be useful as an air passenger demand proxy in the UK. To prove this primary objective, this work uses several machine and deep learning multi-layer perceptron (MLP) methods based on a big-data framework. The findings indicate that DI is a crucial predictor of the UK air passenger demand. Besides, popular error metrics (RMSE, MAPE, MAD and AIC) were compared to find the best model in this study. Specifically, results indicate that MLP following feed forward neural networks works better for the UK air passenger market.

ANFIS learning using expectation maximization based Gaussian mixture model and multilayer perceptron learning

The Adaptive Neuro-Fuzzy Inference System (ANFIS) is a hybrid learning algorithm that combines the learning ability of neural networks with fuzzy inference systems. While ANFIS has been successfully applied to several real-world problems, effective parameter optimization remains a challenge. This paper presents a novel approach for optimizing parameters of ANFIS in supervised settings. The proposed approach utilizes a combination of probabilistic mixture models and perceptron-based learning to parameterize ANFIS. To learn ANFIS membership functions, it generates a mixture of the finite probability distributions for each input feature. Then the consequent parameters are learned by transforming them into weights of a multi-layer perceptron instance. The effectiveness of the proposed method is evaluated on classification problems of varying complexity, ranging from binary-class to multi-class with variable dimensions. The results show that the proposed algorithm improves ANFIS performance in terms of accuracy rate and speed (train-time) analysis. The effectiveness and efficiency of the proposed approach have been further confirmed using a 5 × 2 cross-validation paired significance t-test. In particular, for binary-class problems, our model outperformed standard methods by up to 10% accuracy improvement. Similarly, for multi-class problems, our model achieved an average increase of 8% in accuracy while reaching up to 14% improvement. On average, the proposed model showed a reduction of about 75% in training time compared to the other models. Overall, the proposed approach offers competitive computational performance and acceptable efficacy for parameter optimization of ANFIS.

Sparse Pressure-Based Machine Learning Approach for Aerodynamic Loads Estimation During Gust Encounters

Estimation of aerodynamic loads is a significant challenge in complex gusty environments due to the associated complexities of flow separation and strong nonlinearities. In this study, we explore the practical feasibility of multilayer perceptron (MLP) for estimating aerodynamic loads in gusts, when confounded by noisy and spatially distributed sparse surface pressure measurements. As a demonstration, a nonslender delta wing experiencing various gusts with different initial and final conditions is considered. Time-resolved lift and drag, and spatially distributed sparse surface pressure measurements are collected in a towing-tank facility. The nonlinear MLP model is used to estimate gust scenarios that are unseen in training progress. A filtering process allows us to examine the fluctuation of the dynamic response from the pressure measurements on the MLP. Estimation results show that the MLP model is able to capture the relationship between surface pressure and aerodynamic loads with a minimum quantity of learning samples, delivering accurate estimations, despite the slightly large errors for the cases at the boundary of the datasets. The results also indicate that the dynamic response of the pressure measurements has an influence on the learning of MLP. We further utilize gradient maps to perform a sensitivity analysis, so as to evaluate the contribution of the pressure data to the estimation of gust loads. This study reveals the significant contribution of the sensors located near the leading edge and at the nose of the delta wing. Our findings suggest the potential for an efficient sensor deployment strategy in data-driven aerodynamic load estimation.

An efficient convergence-boosted salp swarm optimizer-based artificial neural network for the development of software fault prediction models

Machine learning (ML) approaches were employed to tackle the software fault prediction (SFP) issue due to their consistent and rigorous performance. Multilayer perceptron (MLP) neural networks are one of the most effective ML models for prediction problems. Unfortunately, MLP suffers from chronic shortcomings related to the gradient-descent learning mechanism that can easily get stuck in local minima, leading to inappropriate control parameters. For SFP tackling, MLP boosted with an improved version of Salp Swarm Optimizer (SSA), a metaheuristic swarm intelligence technique, is presented in this research. The MLP learning approach is updated with SSA to remedy these flaws. The key benefit of such an algorithm is its ability to avoid local minima through convergence behavior. Two improvements were developed in the SSA optimization loop to link SSA functionality with MLP. The first improvement is elitism (SSA-elitism), while the second is MSSA, which stands for search improvement. The performance of proposed SSA versions is evaluated using 18 benchmark SFP datasets using ROC, sensitivity, specificity, and accuracy performance metrics. Several evaluations and validations were performed by contrasting the results of the developed versions with those of the conventional MLP, SSA, and 10 state-of-the-art approaches. The evaluations and validation findings prove that the proposed versions have superior capabilities to efficiently optimize MLP parameters, which raises the quality of their predictability.

Deep learning model on rates of change for multi-step ahead streamflow forecasting

Abstract Water security and urban flooding have become major sustainability issues. This paper presents a novel method to introduce rates of change as the state-of-the-art approach in artificial intelligence model development for sustainability agenda. Multi-layer perceptron (MLP) and deep learning long short-term memory (LSTM) models were considered for flood forecasting. Historical rainfall data from 2008 to 2021 at 11 telemetry stations were obtained to predict flow at the confluence between Klang River and Ampang River. The initial results of MLP yielded poor performance beneath normal expectations, which was R = 0.4465, MAE = 3.7135, NSE = 0.1994 and RMSE = 8.8556. Meanwhile, the LSTM model generated a 45% improvement in its R-value up to 0.9055. Detailed investigations found that the redundancy of data input that yielded multiple target values had distorted the model performance. Qt was introduced into input parameters to solve this issue, while Qt+0.5 was the target value. A significant improvement in the results was detected with R = 0.9359, MAE = 0.7722, NSE = 0.8756 and RMSE = 3.4911. When the rates of change were employed, an impressive improvement was seen for the plot of actual vs. forecasted flow. Findings showed that the rates of change could reduce forecast errors and were helpful as an additional layer of early flood detection.

Voltage sag source location based on multi-layer perceptron and transfer learning

The existing voltage sag source localization only utilizes one simulation type or measurement data based on sampled waveform data. Moreover, it requires more storage space and transmission channels and cannot combine the advantages of simulated and measurement data, resulting in poor applicability of the model. Hence, this paper proposes a voltage sag source locating method based on transfer learning from the sag information in the sag event list combined with the grid structure data. Firstly, the location features of the sag source are extracted from the degree of sag impact, network structure, and sag-type information based on the characteristics of simulated and measurement data that characterize the position of the sag source, and they are collectively used as inputs to the model. Then, the simulated data is used to build a multi-classification model based on the multi-layer perceptron with the line number as the classification number, and the measurement data is employed to fine-tune the model parameters to achieve transfer learning. Finally, voltage sag source localization is achieved based on the trained multi-classification model. The correctness of the proposed method in this paper is verified through simulation and actual measurement in a specific area of East China.

Machine learning multilayer perceptron method for building information modeling application in engineering performance prediction

ABSTRACT The engineering design process has fundamentally impacted the life cycle of construction projects and notably, the engineering performance is significantly measured in delivering projects. Previous studies on engineering performance have established the cause–effect relationships between project variables and performance measures. Recently, the building information modeling (BIM) application has reformed how owners execute the engineering, construction, commissioning, and operation in the industry. There has been an increasing focus in finding the benefits of BIM on project performance, however, a minor focus has been given to engineering performance. This paper proposes an artificial neural network (ANN) machine learning multilayer perceptron (MLMP) method and linear regression (LR) that correlates the use of BIM with engineering performance for better construction project assessment. The conclusions reveal a high-level correlation measure between BIM use inputs and engineering performance outputs and further methods for evaluating the engineering performance. Furthermore, we achieved and validated the best prediction by leveraging data from 60 samples using the MLMP and LR models.

A Multilayer Approach for Intrusion Detection with Lightweight Multilayer Perceptron and LSTM Deep Learning Models

Intrusion detection is essential in the field of cybersecurity for protecting networks and computer systems from nefarious activity. We suggest a novel multilayer strategy that combines the strength of the Lightweight Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM) deep learning models in order to improve the precision and effectiveness of intrusion detection.The initial layer for extraction of features and representation is the Lightweight MLP. Its streamlined architecture allows for quick network data processing while still maintaining competitive performance. The LSTM deep learning model, which is excellent at identifying temporal correlations and patterns in sequential data, receives the extracted features after that.Our multilayer technique successfully manages the highly dimensional and dynamic nature of data from networks by merging these two models. We undertake extensive tests on benchmark datasets, and the outcomes show that our strategy performs better than conventional single-model intrusion detection techniques.The suggested multilayer method also demonstrates outstanding efficiency, which makes it particularly ideal for real-time intrusion detection in expansive network environments. Our multilayer approach offers a strong and dependable solution for identifying and reducing intrusions, strengthening the security position of computer systems and networks as cyber threats continue to advance.

A comparison of quaternion neural network backpropagation algorithms

This research paper focuses on quaternion neural networks (QNNs) - a type of neural network wherein the weights, biases, and input values are all represented as quaternion numbers. Previous studies have shown that QNNs outperform real-valued neural networks in basic tasks and have potential in high-dimensional problem spaces. However, research on QNNs has been fragmented, with contributions from different mathematical and engineering domains leading to unintentional overlap in QNN literature. This work aims to unify existing research by evaluating four distinct QNN backpropagation algorithms, including the novel GHR-calculus backpropagation algorithm, and providing concise, scalable implementations of each algorithm using a modern compiled programming language. Additionally, the authors apply a robust Design of Experiments (DoE) methodology to compare the accuracy and runtime of each algorithm. The experiments demonstrate that the Clifford Multilayer Perceptron (CMLP) learning algorithm results in statistically significant improvements in network test set accuracy while maintaining comparable runtime performance to the other three algorithms in four distinct regression tasks. By unifying existing research and comparing different QNN training algorithms, this work develops a state-of-the-art baseline and provides important insights into the potential of QNNs for solving high-dimensional problems.

A short-term voltage stability online assessment based on multi-layer perceptron learning

In this paper, a deep learning-based approach is used to analyze short-term voltage stability of power systems in order to completely learn the hidden timely dependencies from post fault behavior of the system. The direct measurements from the PMU, which did not require any significant amount of computation, have been used, and the results will be accurate. The proposed approach makes considerable use of a Multi-Layer Perceptron-based deep learning model in order to make the most accurate prediction feasible regarding the short-term voltage stability status. Rapid prediction of voltage stability allowed system operators sufficient time to take effective corrective action against large disturbances. In-depth offline simulations will be executed against various fault types, with the results being fed into the classifier as inputs. After the classifier has been trained using the results of the offline simulations, the online application will be applied to the test samples. The effectiveness of the suggested method will be evaluated based on the results of tests conducted on both small and large-scale test systems.

A virtual biopsy study of microsatellite instability in gastric cancer based on deep learning radiomics

ObjectivesThis study aims to develop and validate a virtual biopsy model to predict microsatellite instability (MSI) status in preoperative gastric cancer (GC) patients based on clinical information and the radiomics of deep learning algorithms.MethodsA total of 223 GC patients with MSI status detected by postoperative immunohistochemical staining (IHC) were retrospectively recruited and randomly assigned to the training (n = 167) and testing (n = 56) sets in a 3:1 ratio. In the training set, 982 high-throughput radiomic features were extracted from preoperative abdominal dynamic contrast-enhanced CT (CECT) and screened. According to the deep learning multilayer perceptron (MLP), 15 optimal features were optimized to establish the radiomic feature score (Rad-score), and LASSO regression was used to screen out clinically independent predictors. Based on logistic regression, the Rad-score and clinically independent predictors were integrated to build the clinical radiomics model and visualized as a nomogram and independently verified in the testing set. The performance and clinical applicability of hybrid model in identifying MSI status were evaluated by the area under the receiver operating characteristic (AUC) curve, calibration curve, and decision curve (DCA).ResultsThe AUCs of the clinical image model in training set and testing set were 0.883 [95% CI: 0.822–0.945] and 0.802 [95% CI: 0.666–0.937], respectively. This hybrid model showed good consistency in the calibration curve and clinical applicability in the DCA curve, respectively.ConclusionsUsing preoperative imaging and clinical information, we developed a deep-learning-based radiomics model for the non-invasive evaluation of MSI in GC patients. This model maybe can potentially support clinical treatment decision making for GC patients.Graphical abstract

Machine Learning for the Prediction of Ionization Potential and Electron Affinity Energies Obtained by Density Functional Theory

AbstractQuantum chemical (QC) calculations based on density functional theory (DFT) provide increasingly accurate estimates of various properties, but with a relatively high computational cost. Machine learning (ML) techniques can be envisaged to extract new knowledge from these large volumes of data, creating empirical models to fast predict QC calculations in new situations. Here, ML algorithms were explored for the fast estimation of ionization potential (IP) and electron affinity (EA) energies calculated by DFT using the B3LYP and PBE0 with 6–31G** basic set on molecular descriptors generated from DFT‐optimized geometries. A database of 9,410 and 9,627 small organic structures for IP and EA energies modelling were used, respectively. Several ML algorithms such as random forest, support vector machines, deep learning multilayer perceptron networks, and light gradient‐boosting machine were screened. The best performance was achieved with a consensus regression model predicted an external test set of 972 and 963 small organic molecules achieving a mean absolute error up to 0.23 eV and 0.32 eV for modelling IP and EA energies, respectively.

Modelling the Economic Growth of Türkiye Dependent on the Sectoral Energy Consumptions, Labour Force and the Capital Formation Using Machine Learning Methods

Energy is an important input of the economic growth and the energy policies of countries are crucial for reaching their economic targets. In this study, the economic growth of Türkiye for the period of 1990-2021 is modelled dependent on the sectoral energy consumption, labour force and the capital formation. The dependent and independent data are taken from the respective sources and then the pairwise Granger causality test is applied on these data. As the next step, the economic growth of Türkiye is taken as the dependent variable and modelled as a function of the other variables. Considering the low number of samples available and the nonlinearity of the data, machine learning methods are utilized for the modelling. Specifically, deep learning multilayer perceptron networks which are coded in Python programming language are employed for the modelling of the economic growth. The 70% of the available data are used as the training data while 30% of the data are utilized as the test data. The training and test data are split using special classes of the Python programming language to provide objectivity. Then, the actual economic growth and the deep learning model results are graphed which show a high degree of overlap indicating the accuracy of the developed model. In addition, the performance metrics of the developed model namely the coefficient of determination, mean absolute error, mean absolute percentage error and the root mean square error are computed which also indicate the high accuracy of the developed model. The approach used for the modelling of the economic growth is considered to be useful for modelling the other econometric data related to the growth hypothesis.

Prediction and optimization of sustainable fuel cells behavior using artificial intelligence algorithms

The model and optimization of parameters have been performed using a computer intelligence approach in order to provide greater convenience and reduce costs. In this study, the power density and voltage behavior of 13 direct glucose fuel cells (three types: MEA1-MEA4, MEA5-MEA8, and MEA10-MEA14) are modeled through response surface methodology, multi-layer perceptron (MLP), and machine learning (ML). First, the regression coefficients (R2) of the power density RSM model for MEA1-MEA4, MEA5-MEA8, and MEA10-MEA14 are calculated to be 0.99839, 0.99529, and 0.95316, respectively. After that, the best MLP-ANN structure for predicting power density and the voltage of fuel cells is selected with two hidden layers, which have 5, 5 neurons in each hidden layer for MEA1-MEA4, 3, 5 neurons for MEA5-MEA8, and 5, 5 neurons for MEA10-MEA14. Next, multi-objective optimization for power density and the voltage of the fuel cells is performed using three different algorithms: RSM, non-dominated sorting genetic algorithm version 2 (NSGA-II), and the Strength Pareto Evolutionary Algorithm 2 (SPEA2). Finally, the best optimum mode of each type of direct glucose fuel cell is investigated by RSM.

Experimental comparison of R290 and R600a and prediction of performance with machine learning algorithms

The use of alternative refrigerants is among the popular topics of the refrigeration industry. In the first part of this study, thermodynamic performances of R290 and R600a gases were compared in a vapor compression refrigeration experiment setup. Although R600a caused an average of 33.44% less compressor power consumption compared to R290 refrigerant, R290 provided an average of 23.77% increase in COP (coefficient of performance), 82.55% in cooling capacity, and 20.99% increase in second law efficiency compared to R600a. In the second part of the study, the performance parameters of the refrigeration system were predicted with MLP (multi-layer perceptron), SVM (support vector machine), and DT (decision tree) machine learning algorithms. It was detected that the SVM method predicted all parameters with the least error. MAE (mean absolute error) values detected in the COP prediction with test set were 0.0317, 0.0324, and 0.0989 for SVM, MLP, and DT, respectively. Results revealed that performance of the refrigeration system increased when utilizing R290, and SVM was superior in prediction of performance indicators compared to other machine learning methods.

A Study on ML-Based Software Defect Detection for Security Traceability in Smart Healthcare Applications

Software Defect Prediction (SDP) is an integral aspect of the Software Development Life-Cycle (SDLC). As the prevalence of software systems increases and becomes more integrated into our daily lives, so the complexity of these systems increases the risks of widespread defects. With reliance on these systems increasing, the ability to accurately identify a defective model using Machine Learning (ML) has been overlooked and less addressed. Thus, this article contributes an investigation of various ML techniques for SDP. An investigation, comparative analysis and recommendation of appropriate Feature Extraction (FE) techniques, Principal Component Analysis (PCA), Partial Least Squares Regression (PLS), Feature Selection (FS) techniques, Fisher score, Recursive Feature Elimination (RFE), and Elastic Net are presented. Validation of the following techniques, both separately and in combination with ML algorithms, is performed: Support Vector Machine (SVM), Logistic Regression (LR), Naïve Bayes (NB), K-Nearest Neighbour (KNN), Multilayer Perceptron (MLP), Decision Tree (DT), and ensemble learning methods Bootstrap Aggregation (Bagging), Adaptive Boosting (AdaBoost), Extreme Gradient Boosting (XGBoost), Random Forest(RF), and Generalized Stacking (Stacking). Extensive experimental setup was built and the results of the experiments revealed that FE and FS can both positively and negatively affect performance over the base model or Baseline. PLS, both separately and in combination with FS techniques, provides impressive, and the most consistent, improvements, while PCA, in combination with Elastic-Net, shows acceptable improvement.

Estimating Land Subsidence and Gravimetric Anomaly Induced by Aquifer Overexploitation in the Chandigarh Tri-City Region, India by Coupling Remote Sensing with a Deep Learning Neural Network Model

This study utilizes surface displacement data from Persistent Scatterer SAR Interferometry (PSInSAR) of Sentinel-1 satellite and groundwater storage change data from the Gravity Recovery and Climate Experiment (GRACE) satellite mission to understand land subsidence in the Chandigarh tri-city region. The satellite datasets are used along with the groundwater level data obtained from wells over the study area. Since the GRACE data are available at a much coarser spatial resolution of 1o by 1o, challenges remain in correlating the dataset with PSInSAR displacement that has been multi-looked at 14 m by 14 m resolution. Therefore, multiple sources of data (i.e., the monthly average of GRACE data, groundwater storage change and monthly average PSInSAR displacement per pixel, and interpolated groundwater level data from wells for 2017 to 2022) have been deployed into a deep learning multi-layer perceptron (DLMLP) model to estimate the groundwater storage change at the urban level. This has an indirect downscaling method that is carried out successfully using the DLMLP model for the estimation of groundwater storage changes at the urban level, which is usually complicated by applying direct downscaling methods on the GRACE data. Thus, the DLMLP model developed here is a distinctive approach considered for estimating the changes in groundwater storage using PSInSAR displacement, groundwater data from wells, and GRACE data. The DLMLP model gives an R2-statistics value of 0.91 and 0.89 in the training and testing phases, respectively, and has a mean absolute error (MAE) of 1.23 and root mean square error (RMSE) of 0.87.

GANDSE: Generative Adversarial Network-based Design Space Exploration for Neural Network Accelerator Design

With the popularity of deep learning, the hardware implementation platform of deep learning has received increasing interest. Unlike the general purpose devices, e.g., CPU or GPU, where the deep learning algorithms are executed at the software level, neural network hardware accelerators directly execute the algorithms to achieve higher energy efficiency and performance improvements. However, as the deep learning algorithms evolve frequently, the engineering effort and cost of designing the hardware accelerators are greatly increased. To improve the design quality while saving the cost, design automation for neural network accelerators was proposed, where design space exploration algorithms are used to automatically search the optimized accelerator design within a design space. Nevertheless, the increasing complexity of the neural network accelerators brings the increasing dimensions to the design space. As a result, the previous design space exploration algorithms are no longer effective enough to find an optimized design. In this work, we propose a neural network accelerator design automation framework named GANDSE, where we rethink the problem of design space exploration, and propose a novel approach based on the generative adversarial network (GAN) to support an optimized exploration for high-dimension large design space. The experiments show that GANDSE is able to find the more optimized designs in negligible time compared with approaches including multilayer perceptron and deep reinforcement learning.

An Intelligent Network Traffic Prediction Scheme Based on Ensemble Learning of Multi-Layer Perceptron in Complex Networks

At present, the amount of network equipment, servers, and network traffic is increasing exponentially, and the way in which operators allocate and efficiently utilize network resources has attracted considerable attention from traffic forecasting researchers. However, with the advent of the 5G era, network traffic has also shown explosive growth, and network complexity has increased dramatically. Accurately predicting network traffic has become a pressing issue that must be addressed. In this paper, a multilayer perceptron ensemble learning method based on convolutional neural networks (CNN) and gated recurrent units (GRU) spatiotemporal feature extraction (MECG) is proposed for network traffic prediction. First, we extract spatial and temporal features of the data by convolutional neural networks (CNN) and recurrent neural networks (RNN). Then, the extracted temporal features and spatial features are fused into new spatiotemporal features through integrated learning of a multilayer perceptron, and a spatiotemporal prediction model is built in the sequence-to-sequence framework. At the same time, the teacher forcing mechanism and attention mechanism are added to improve the accuracy and convergence speed of the model. Finally, the proposed method is compared with other deep learning models for experiments. The experimental results show that the proposed method not only has apparent advantages in accuracy but also shows some superiority in time training cost.