Multilayer Perceptron Regression Research Articles

Coarse to superfine: can hyperspectral soil organic carbon models predict higher-resolution information?

Introduction: Modeling and mapping of soil organic carbon concentration and distribution at the pedon scale is a current knowledge gap that can be addressed by laboratory-based hyperspectral imaging and chemometric analysis of soil cores. Despite the advancements in soil organic carbon models based on hyperspectral images, it is not clear how these models will perform upon input with images at higher resolutions than those of their training sets. This study aims to measure the generalizability of a soil organic carbon model based on a test set with higher resolution hyperspectral images than that of its training set.Methods: Organic carbon contents were measured at 10 cm intervals on eight soil cores for use as the training set and at 1 cm intervals on a single core for use as the test set. Three regression models, namely, multilayer perceptron, partial least-squares, and support vector regressions, were trained and tested with the median of each hyperspectral image for each of these intervals as the training and test predictors. Permutation importance analysis was performed to explain the models.Results: The results show that although all three models had the same validation R2 of 0.92 for cross-validation on the 10 cm data, multilayer perceptron regression allowed the best generalization with a test R2 of 0.96 compared to the partial least-squares regression (0.81) and support vector regression (0.86). It was demonstrated that the multilayer perceptron model is more robust to soil surface anomalies and that it predicts soil organic carbon on the test set by learning the spectral features related to soil organic matter chromophore activity in the 950–1,150 nm region along with clay mineralogy derived from peaks at 1,400, 1,900, 2,200, 2,250, and 2,350 nm.Conclusions: This study shows that while the regression models based on hyperspectral images perform well at the 10-cm-resolution cross validation, multilayer perceptron regression shows superior generalization and robustness for a higher 1-cm-resolution test set without much loss of prediction power.

Estimating built-up area carbon emissions through addressing regional development disparities with population and nighttime light data

Carbon emission inventories in urban built-up areas are critical for formulating effective emission reduction policies. Nighttime light data, reflecting socioeconomic activities, is routinely used as a proxy for estimating emissions. However, the reliability of this proxy is limited in areas with minimal or absent lighting and demonstrates variability according to the urban development. To address these limitations, a novel approach was developed. Firstly, it enhances the representation of human activities under different light values through integrating light values and population distributions into multidimensional proxy features. Secondly, cities with different socio-economic development were clustered to minimize the development disparities within the same cluster. Four clusters were detected including ‘Agriculture-dependent & Low-density’, ‘Agriculture-dependent & High-density’, ‘Industry-oriented’, and ‘Tertiary-oriented’. Then A multilayer perceptron regressor, refined with a moving average method, was employed to model the nonlinear relationships between emissions and these proxy features. The results show a significant increase in modeling accuracy when regional development disparities are incorporated through this clustering strategy. Additionally, the Shapley Additive Explanatory method was utilized to analyze the influence of regional development heterogeneity on the estimation model. It demonstrated that areas with high light intensity and population significantly influence emission estimates in all clusters, except for the ‘Industry-oriented cluster’. This finding underscores the varying effectiveness of proxy features across different regions, emphasizing the importance of considering regional development disparities. Finally, based on the estimated inventory obtained, spatio-temporal hot spot detection pinpointed eastern and central China as significant emission hot spots. This research not only enhances the data resources for managing carbon emissions but also underscores the need for inter-city cooperation strategies to achieve China's carbon reduction targets.

Coupling the PROSAIL Model and Machine Learning Approach for Canopy Parameter Estimation of Moso Bamboo Forests from UAV Hyperspectral Data

Parameters such as the leaf area index (LAI), canopy chlorophyll content (CCH), and canopy carotenoid content (CCA) are important indicators for evaluating the ecological functions of forests. Currently, rapidly developing unmanned aerial vehicles (UAV) equipped with hyperspectral technology provide advanced technical means for the real-time dynamic acquisition of regional vegetation canopy parameters. In this study, a hyperspectral sensor mounted on a UAV was used to acquire the data in the study area, and the canopy parameter estimation model of moso bamboo forests (MBF) was developed by combining the PROSAIL radiative transfer model and the machine learning regression algorithm (MLRA), inverted the canopy parameters such as LAI, CCH, and CCA. The method first utilized the extended Fourier amplitude sensitivity test (EFAST) method to optimize the global sensitivity analysis and parameters of the PROSAIL model, and the successive projections algorithm (SPA) was used to screen the characteristic wavebands for the inversion of MBF canopy parameter inversion. Then, the optimized PROSAIL model was used to construct the ‘LAI-CCH-CCA-canopy reflectance’ simulation dataset for the MBF; multilayer perceptron regressor (MLPR), extra tree regressor (ETR), and extreme gradient boosting regressor (XGBR) employed used to construct PROSAIL_MLPR, PROSAIL_ETR, and PROSAIL_XGBR, respectively, as the three hybrid models. Finally, the best hybrid model was selected and used to invert the spatial distribution of the MBF canopy parameters. The following results were obtained: Waveband sensitivity analysis reveals 400–490 and 710–1000 nm as critical for LAI, 540–650 nm for chlorophyll, and 490–540 nm for carotenoids. SPA narrows down the feature bands to 43 for LAI, 19 for CCH, and 9 for CCA. The three constructed hybrid models were able to achieve high-precision inversion of the three parameters of the MBF, the model fitting accuracy of PROSAIL_MLRA reached more than 95%, with lower RMSE values, and the PROSAIL_XGBR model yielded the best fitting results. Our study provides a novel method for the inversion of forest canopy parameters based on UAV hyperspectral data.

Forecasting of energy efficiency in buildings using multilayer perceptron regressor with waterwheel plant algorithm hyperparameter

Energy consumption in buildings is gradually increasing and accounts for around forty percent of the total energy consumption. Forecasting the heating and cooling loads of a building during the initial phase of the design process in order to identify optimal solutions among various designs is of utmost importance. This is also true during the operation phase of the structure after it has been completed in order to ensure that energy efficiency is maintained. The aim of this paper is to create and develop a Multilayer Perceptron Regressor (MLPRegressor) model for the purpose of forecasting the heating and cooling loads of a building. The proposed model is based on automated hyperparameter optimization using Waterwheel Plant Algorithm The model was based on a dataset that described the energy performance of the structure. There are a number of important characteristics that are considered to be input variables. These include relative compactness, roof area, overall height, surface area, glazing area, wall area, glazing area distribution of a structure, and orientation. On the other hand, the variables that are considered to be output variables are the heating and cooling loads of the building. A total of 768 residential buildings were included in the dataset that was utilized for training purposes. Following the training and regression of the model, the most significant parameters that influence heating load and cooling load have been identified, and the WWPA-MLPRegressor performed well in terms of different metrices variables and fitted time.

Acoustic impedance prediction based on extended seismic attributes using multilayer perceptron, random forest, and extra tree regressor algorithms

AbstractAcoustic impedance is the product of the density of a material and the speed at which an acoustic wave travels through it. Understanding this relationship is essential because low acoustic impedance values are closely associated with high porosity, facilitating the accumulation of more hydrocarbons. In this study, we estimate the acoustic impedance based on nine different inputs of seismic attributes in addition to depth and two-way travel time using three supervised machine learning models, namely extra tree regression (ETR), random forest regression, and a multilayer perceptron regression algorithm using the scikit-learn library. Our results show that the R2 of multilayer perceptron regression is 0.85, which is close to what has been reported in recent studies. However, the ETR method outperformed those reported in the literature in terms of the mean absolute error, mean squared error, and root-mean-squared error. The novelty of this study lies in achieving more accurate predictions of acoustic impedance for exploration.

New paradigm for watershed model development by coupling machine learning algorithm and mechanistic model

The development of watershed models faces double pressure of data requirements and physical interpretation. The simulation of nonpoint source (NPS) pollution is an important application of watershed model, and the best management practices (BMPs) have attracted wide attention as the main control approach of NPS pollution. In this study, a new paradigm was proposed based on the integration of data-driven and mechanistic methods, taking BMP evaluation as an example. Specifically, comprehensive databases were constructed for filter and retention BMPs by collecting, classifying and analyzing published data. Twelve machine learning algorithms were employed for regression analysis between BMPs efficiency and their influencing factors, while the constructed equations were coupled with physical-based models. A case study was performed in a typical catchment of Chaohu Lake Watershed, China. The results demonstrated total interception area, soil type, and vegetation biomass had significant impacts on BMPs performances, while the multilayer perceptron regression (MLPR), K nearest neighbor regression (KNRD), and extremely randomized tree (ETR) methods had the best performances of nutrients removal. The study generated over ten thousand datasets using mechanistic processes, resulting in more efficient and interpretable BMPs evaluator. Compare to traditional methods, this new paradigm has shown advantages in the model application at the watershed scale, enhancing the BMPs evaluator with a higher level of interpretability by coupling the approach with mechanistic models.

Volve Oil Field S-Wave Log Data Prediction Using GBR and MLPR

The shear wave sonic (S-wave) log data is essential for identifying the reservoir's geomechanical properties, which is an important factor for the drilling, completion, and optimization processes, where obtaining S-wave requires capital investment and reduces the cost. In this paper, we used the multi-layer perceptron (MLPR) and the gradient boosting regression (GBR) to predict the missing S-Wave log data for the Volve Oil Field in the North Sea. Prescriptive and predictive analysis were carried out in series to achieve a high blind data accuracy rate of 0.943 and 0.982 for the multi-layer perceptron regression and the gradient boosting regression, respectively. It was observed that the gradient-boosting algorithm achieved higher accuracy than the MLPR algorithm for the limited dataset. It was also found that hierarchical clustering can reveal information regarding the feature's importance similar to the relevant AI tools, which makes hierarchical clustering a faster tool in eliminating the nonimportant inputs from the dataset while the use of artificial intelligence tools showed a significant effect in predicting the missing values of the sonic wave log and the neutron porosity log in an efficient way by selecting the relevant important features.

YOUNG’S MODULUS OF CALCIUM-ALUMINO-SILICATE GLASSES: INSIGHT FROM MACHINE LEARNING

Modern technologies require the development of new materials with exceptional properties. Machine Learning (ML) and Deep Learning (DL) techniques have become important tools for discovering new materials and predicting the properties of specific materials, such as glasses. In this paper, we used ML and DL techniques to predict the Young's modulus E of Calcium-Alumino-Silicate (CAS) glasses based on their chemical composition. We evaluated four different algorithms, including Polynomial Regression (PR), Random Forest (RF), K-Nearest Neighbors (KNN), and Multi-Layer Perceptron Regressor (MLPRegressor). We found that the PR algorithm provides excellent predictions without Cross-Validation (CV), while the MLPRegressor yields the best performance when CV is implemented.

Predicting Crime rate in Punjab: An Analysis of Artificial Intelligence based Models

This study aims at measuring and evaluating the accuracy rate or Mean Square Error (MSE) of different machine learning models to predict crime count (dependent variable) on the basis of a number of independent variables such as shift, day, month, police station, district and time of the day. It is a quantitative research study where primary data is collected from rescue 15 call record. The year 2022 has been chosen for the purpose of data under the crime head of theft across the Punjab province which is divided into three eight hourly shifts each in a day during any week of a month. The crime count indicates number of incidents of theft reported during these three shifts per day per month in a district in 2022. The data containing variables is then used for prediction through an appropriate machine learning model. The MSE value of Random Forest Regression (RFR) is greater than MSE value of Multi-Layer Perceptron Regression (MLPR). It is however lower than MSE score of Linear Regression (LR) model and greater than MSE value of Support Vector Regression (SVR). This shows that MLPR model is the most suitable one with least error as compared to other models which have higher MSE values while predicting the dependent variable (theft crime count). The study findings may assist the intelligence branches of police to efficiently utilize their material and human resources while applying the appropriate model of machine learning to predict the number of crime incidents in any geographical dispensation at sub national level.

Comparative Analysis Using Multiple Regression Models for Forecasting Photovoltaic Power Generation

Effective machine learning regression models are useful toolsets for managing and planning energy in PV grid-connected systems. Machine learning regression models, however, have been crucial in the analysis, forecasting, and prediction of numerous parameters that support the efficient management of the production and distribution of green energy. This article proposes multiple regression models for power prediction using the Sharda University PV dataset (2022 Edition). The proposed regression model is inspired by a unique data pre-processing technique for forecasting PV power generation. Performance metrics, namely mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), R2-score, and predicted vs. actual value plots, have been used to compare the performance of the different regression. Simulation results show that the multilayer perceptron regressor outperforms the other algorithms, with an RMSE of 17.870 and an R2 score of 0.9377. Feature importance analysis has been performed to determine the most significant features that influence PV power generation.

Enhancement of multirotor UAV conceptual design through Machine Learning algorithms

Designing an efficient and optimized multirotor UAV requires laborious trade-off analyses, involving numerous design variables and mission requirement parameters, especially during the early conceptual design phase. The large number of unknown parameters, as well as the associated design effort often leads to non-optimal designs, for the sake of time efficiency. This work presents the implementation of a machine learning (ML) framework to assist and expedite the conceptual design phase of multirotor UAVs. The framework utilizes information from a comprehensive database of commercial lightweight multirotor UAVs. The database contains an extensive collection of crucial sizing parameters, performance metrics, and features associated with foldability and indoor guidance (e.g., obstacle avoidance sensors). These attributes specifically pertain to multirotor UAVs weighing less than 2kg, which exhibit diverse design and performance characteristics. The proposed ML framework employs multiple regression models (e.g. k-nearest neighbors regression, multi-layer perceptron regression) to predict the sizing parameters during a multirotor UAV’s conceptual design phase. This enables designers to make quick informed decisions, while also significantly reducing computational time and effort. Finally, the ML framework’s predictive capability is validated by comparing the predicted values with real-world data from an “unseen” test dataset.

Leveraging Machine Learning for Fraudulent Social Media Profile Detection

Abstract Fake social media profiles are responsible for various cyber-attacks, spreading fake news, identity theft, business and payment fraud, abuse, and more. This paper aims to explore the potential of Machine Learning in detecting fake social media profiles by employing various Machine Learning algorithms, including the Dummy Classifier, Support Vector Classifier (SVC), Support Vector Classifier (SVC) kernels, Random Forest classifier, Random Forest Regressor, Decision Tree Classifier, Decision Tree Regressor, MultiLayer Perceptron classifier (MLP), MultiLayer Perceptron (MLP) Regressor, Naïve Bayes classifier, and Logistic Regression. For a comprehensive evaluation of the performance and accuracy of different models in detecting fake social media profiles, it is essential to consider confusion matrices, sampling techniques, and various metric calculations. Additionally, incorporating extended computations such as root mean squared error, mean absolute error, mean squared error and cross-validation accuracy can further enhance the overall performance of the models.

Machine Learning Accelerated Discovery of Functional MXenes with Giant Piezoelectric Coefficients.

Efficient and rapid screening of target materials in a vast material space remains a significant challenge in the field of materials science. In this study, first-principles calculations and machine learning algorithms are performed to search for high out-of-plane piezoelectric stress coefficient materials in the MXene functional database among the 1757 groups of noncentrosymmetric MXenes with nonzero band gaps, which meet the criteria for piezoelectric properties. For the monatomic MXene testing set, the random forest regression (RFR), gradient boosting regression (GBR), support vector regression (SVR), and multilayer perceptron regression (MLPR) exhibit R2 values of 0.80, 0.80, 0.89, and 0.87, respectively. Expanding our analysis to the entire MXene data set, the best active learning cycle finds more than 140 and 22 MXenes with out-of-plane piezoelectric stress coefficients (e31) exceeding 3 × 10-10 and 5 × 10-10 C/m, respectively. Moreover, thermodynamic stabilities were confirmed in 22 MXenes with giant piezoelectric stress coefficients and 9 MXenes with both large in-plane (d11 > 15 pm/V) and out-of-plane (d31 > 2 pm/V) piezoelectric strain coefficients. These findings highlight the remarkable capabilities of machine learning and its optimization algorithms in accelerating the discovery of novel piezoelectric materials, and MXene materials emerge as highly promising candidates for piezoelectric materials.

Trajectory Predictor and Conflict Detection Figures of Merit for a Performance-Based Adaptive Air Traffic Monitoring System

This paper investigates the impact of trajectory predictor performance on the encounter probability generated by an adaptive conflict detection tool and examines the flexibility of the tool dependent on its adjustable thresholds, using historical radar track data. To achieve these objectives, two figures of merit were proposed: System Dynamic Range and System Tuning Envelope. To examine the conflict detection’s performance variability under different uncertainty levels and predictor types, simple multi-horizon trajectory predictors trained with two machine learning techniques of different characteristics are assessed at various look-ahead times: extreme gradient boosting with a discrete nature and a multi-layer perceptron regressor with a continuous nature. The results highlight the interdependence between the performances of the trajectory predictor and the conflict detector, and the quantification of this relationship can be represented through a sigmoid function. In addition, the two proposed figures of merit are effective for selecting suitable operating points in an adaptive conflict detector, based on dynamic thresholds and the performance requirements necessary for the trajectory predictors to achieve the expected detection performance at different look-ahead time.

Deep learning and big data mining for Metal–Organic frameworks with high performance for simultaneous desulfurization and carbon capture

Carbon capture and desulfurization of flue gases are crucial for the achievement of carbon neutrality and sustainable development. In this work, the “one-step” adsorption technology with high-performance metal–organic frameworks (MOFs) was proposed to simultaneously capture the SO2 and CO2. Four machine learning algorithms were used to predict the performance indicators (NCO2+SO2, SCO2+SO2/N2, and TSN) of MOFs, with Multi-Layer Perceptron Regression (MLPR) showing better performance (R2 = 0.93). To address sparse data of MOF chemical descriptors, we introduced the Deep Factorization Machines (DeepFM) model, outperforming MLPR with a higher R2 of 0.95. Then, sensitivity analysis was employed to find that the adsorption heat and porosity were the key factors for SO2 and CO2 capture performance of MOF, while the influence of open alkali metal sites also stood out. Furthermore, we established a kinetic model to batch simulate the breakthrough curves of TOP 1000 MOFs to investigate their dynamic adsorption separation performance for SO2/CO2/N2. The TOP 20 MOFs screened by the dynamic performance highly overlap with those screened by the static performance, with 76 % containing open alkali metal sites. This integrated approach of computational screening, machine learning, and dynamic analysis significantly advances the development of efficient MOF adsorbents for flue gas treatment.

Improvement of Machine Learning-Based Modelling of Container Ship’s Main Particulars with Synthetic Data

One of the main problems in the application of machine learning techniques is the need for large amounts of data necessary to obtain a well-generalizing model. This is exacerbated for studies in which it is not possible to access large amounts of data—for example, in the case of ship main data modelling, where a limited amount of real-world data (ship main data) is available for dataset creation. In this paper, a synthetic data generation technique has been applied to generate a large amount of synthetic data points regarding container ships’ main particulars. Models are trained using a multilayer perceptron (MLP) regressor on both original and synthetic data mixed with original data points. Then, the authors validate the performance of the obtained models on the original data and conclude whether a synthetic-data-based approach can be used to develop models in instances where the amount of data on ship main particulars may be limited. The results demonstrate an improvement across almost all outputs, ranging between 0.01 and 0.21 when evaluated using the coefficient of determination (R2) and between 0.27% and 3.43% when models are evaluated with mean absolute percentage error (MAPE). This indicates that the application of synthetic data can indeed be used for the improvement of ML-based model performance. The presented study demonstrates that the application of ML-based syncretization techniques can provide significant improvements to the process of ML-based determination of a ship’s main particulars at the early design stage. This paper suggests that, in cases where only a small dataset is available, artificial neural networks (ANN) can still be effectively employed to derive early-stage design values for the main particulars through the use of synthetic data.

Prediction of the water level at the Kien Giang River based on regression techniques

AbstractModel accuracy and runtime are two key issues for flood warnings in rivers. Traditional hydrodynamic models, which have a rigorous physical mechanism for flood routine, have been widely adopted for water level prediction in river, lake, and urban areas. However, these models require various types of data, in‐depth domain knowledge, experience with modeling, and intensive computational time, which hinders short‐term or real‐time prediction. In this paper, we propose a new framework based on machine learning methods to alleviate the aforementioned limitation. We develop a wide range of machine learning models such as linear regression (LR), support vector regression (SVR), random forest regression (RFR), multilayer perceptron regression (MLPR), and light gradient boosting machine regression (LGBMR) to predict the hourly water level at Le Thuy and Kien Giang stations of the Kien Giang river based on collected data of 2010, 2012, and 2020. Four evaluation metrics, that is, R2, Nash–Sutcliffe efficiency, mean absolute error, and root mean square error, are employed to examine the reliability of the proposed models. The results show that the LR model outperforms the SVR, RFR, MLPR, and LGBMR models.

Optimizing Neural Network Efficiency by Neuron Pruning

This study primarily focuses on two main objectives. The primary objective is to develop an appropriate model for accurately predicting the missing shear wave data in the Volve Oil Field located in the North Sea. By employing the Multi-Layer Perceptron Regression and modifying the neural network structure, the first goal attains a prediction accuracy of 0.943 for missing S-Wave log data. Furthermore, the objective of the study is to enhance the precision of forecasting incomplete S-Wave log data by optimising the structure of the artificial neural network, using neuron pruning techniques based on sensitivity analysis. This optimisation leads to a heightened accuracy rate of 0.9609. The effectiveness of these pruning strategies is clearly evident in their demonstrated capacity to improve the accurate prediction of missing data in the sonic wave log.

Analysis of heat transfer and prediction of preheating time of snow-melting on a highway T-beam bridge deck

Electrical heating is an innovative method of preventing icing of bridge decks during winter. Turning on the heating system before snowfall to prevent the icing of the bridge deck is more advantageous than melting already-formed ice. This article analyzes the heat-transfer performance of a bridge deck with heating cables during preheating before snowfall and predicts the preheating time of the bridge deck. Firstly, the temperature rise law and the state of the bridge deck with heating cables were studied through field experiments. Subsequently, a numerical model was developed. The effects of four factors on the heat-transfer performance of the bridge deck were analyzed. Finally, a bridge deck preheating-time model was established based on a multilayer perceptron regressor. This study enriches the application of neural networks and numerical simulations in preventing ice formation using the electrical heating method and is important for optimizing the heating system scheme and saving energy.

A Thermo-Structural Analysis of Die-Sinking Electrical Discharge Machining (EDM) of a Haynes-25 Super Alloy Using Deep-Learning-Based Methodologies

The most effective and cutting-edge method for achieving a 0.004 mm precision on a typical material is to employ die-sinking electrical discharge machining (EDM). The material removal rate (MRR), tool wear rate (TWR), residual stresses, and crater depth were analyzed in the current study in an effort to increase the productivity and comprehension of the die-sinking EDM process. A parametric design was employed to construct a two-dimensional model, and the accuracy of the findings was verified by comparing them to prior research. Experiments were conducted utilizing the EDM machine, and the outcomes were assessed in relation to numerical simulations of the MRR and TWR. A significant temperature disparity that arises among different sections of the workpiece may result in the formation of residual strains throughout. As a consequence, a structural model was developed in order to examine the impacts of various stress responses. The primary innovations of this paper are its parametric investigation of residual stresses and its use of Haynes 25, a workpiece material that has received limited attention despite its numerous benefits and variety of applications. In order to accurately forecast the output parameters, a deep neural network model, more precisely, a multilayer perceptron (MLP) regressor, was utilized. In order to improve the precision of the outcomes and guarantee stability during convergence, the L-BFGS solver, an adaptive learning rate, and the Rectified Linear Unit (ReLU) activation function were integrated. Extensive parametric studies allowed us to determine the connection between key inputs, including the discharge current, voltage, and spark-on time, and the output parameters, namely, the MRR, TWR, and crater depth.