Multilayer Perceptron Regression Research Articles

New paradigm for watershed model development by coupling machine learning algorithm and mechanistic model

The development of watershed models faces double pressure of data requirements and physical interpretation. The simulation of nonpoint source (NPS) pollution is an important application of watershed model, and the best management practices (BMPs) have attracted wide attention as the main control approach of NPS pollution. In this study, a new paradigm was proposed based on the integration of data-driven and mechanistic methods, taking BMP evaluation as an example. Specifically, comprehensive databases were constructed for filter and retention BMPs by collecting, classifying and analyzing published data. Twelve machine learning algorithms were employed for regression analysis between BMPs efficiency and their influencing factors, while the constructed equations were coupled with physical-based models. A case study was performed in a typical catchment of Chaohu Lake Watershed, China. The results demonstrated total interception area, soil type, and vegetation biomass had significant impacts on BMPs performances, while the multilayer perceptron regression (MLPR), K nearest neighbor regression (KNRD), and extremely randomized tree (ETR) methods had the best performances of nutrients removal. The study generated over ten thousand datasets using mechanistic processes, resulting in more efficient and interpretable BMPs evaluator. Compare to traditional methods, this new paradigm has shown advantages in the model application at the watershed scale, enhancing the BMPs evaluator with a higher level of interpretability by coupling the approach with mechanistic models.

Volve Oil Field S-Wave Log Data Prediction Using GBR and MLPR

The shear wave sonic (S-wave) log data is essential for identifying the reservoir's geomechanical properties, which is an important factor for the drilling, completion, and optimization processes, where obtaining S-wave requires capital investment and reduces the cost. In this paper, we used the multi-layer perceptron (MLPR) and the gradient boosting regression (GBR) to predict the missing S-Wave log data for the Volve Oil Field in the North Sea. Prescriptive and predictive analysis were carried out in series to achieve a high blind data accuracy rate of 0.943 and 0.982 for the multi-layer perceptron regression and the gradient boosting regression, respectively. It was observed that the gradient-boosting algorithm achieved higher accuracy than the MLPR algorithm for the limited dataset. It was also found that hierarchical clustering can reveal information regarding the feature's importance similar to the relevant AI tools, which makes hierarchical clustering a faster tool in eliminating the nonimportant inputs from the dataset while the use of artificial intelligence tools showed a significant effect in predicting the missing values of the sonic wave log and the neutron porosity log in an efficient way by selecting the relevant important features.

YOUNG’S MODULUS OF CALCIUM-ALUMINO-SILICATE GLASSES: INSIGHT FROM MACHINE LEARNING

Modern technologies require the development of new materials with exceptional properties. Machine Learning (ML) and Deep Learning (DL) techniques have become important tools for discovering new materials and predicting the properties of specific materials, such as glasses. In this paper, we used ML and DL techniques to predict the Young's modulus E of Calcium-Alumino-Silicate (CAS) glasses based on their chemical composition. We evaluated four different algorithms, including Polynomial Regression (PR), Random Forest (RF), K-Nearest Neighbors (KNN), and Multi-Layer Perceptron Regressor (MLPRegressor). We found that the PR algorithm provides excellent predictions without Cross-Validation (CV), while the MLPRegressor yields the best performance when CV is implemented.

Predicting Crime rate in Punjab: An Analysis of Artificial Intelligence based Models

This study aims at measuring and evaluating the accuracy rate or Mean Square Error (MSE) of different machine learning models to predict crime count (dependent variable) on the basis of a number of independent variables such as shift, day, month, police station, district and time of the day. It is a quantitative research study where primary data is collected from rescue 15 call record. The year 2022 has been chosen for the purpose of data under the crime head of theft across the Punjab province which is divided into three eight hourly shifts each in a day during any week of a month. The crime count indicates number of incidents of theft reported during these three shifts per day per month in a district in 2022. The data containing variables is then used for prediction through an appropriate machine learning model. The MSE value of Random Forest Regression (RFR) is greater than MSE value of Multi-Layer Perceptron Regression (MLPR). It is however lower than MSE score of Linear Regression (LR) model and greater than MSE value of Support Vector Regression (SVR). This shows that MLPR model is the most suitable one with least error as compared to other models which have higher MSE values while predicting the dependent variable (theft crime count). The study findings may assist the intelligence branches of police to efficiently utilize their material and human resources while applying the appropriate model of machine learning to predict the number of crime incidents in any geographical dispensation at sub national level.

Comparative Analysis Using Multiple Regression Models for Forecasting Photovoltaic Power Generation

Effective machine learning regression models are useful toolsets for managing and planning energy in PV grid-connected systems. Machine learning regression models, however, have been crucial in the analysis, forecasting, and prediction of numerous parameters that support the efficient management of the production and distribution of green energy. This article proposes multiple regression models for power prediction using the Sharda University PV dataset (2022 Edition). The proposed regression model is inspired by a unique data pre-processing technique for forecasting PV power generation. Performance metrics, namely mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), R2-score, and predicted vs. actual value plots, have been used to compare the performance of the different regression. Simulation results show that the multilayer perceptron regressor outperforms the other algorithms, with an RMSE of 17.870 and an R2 score of 0.9377. Feature importance analysis has been performed to determine the most significant features that influence PV power generation.

Enhancement of multirotor UAV conceptual design through Machine Learning algorithms

Designing an efficient and optimized multirotor UAV requires laborious trade-off analyses, involving numerous design variables and mission requirement parameters, especially during the early conceptual design phase. The large number of unknown parameters, as well as the associated design effort often leads to non-optimal designs, for the sake of time efficiency. This work presents the implementation of a machine learning (ML) framework to assist and expedite the conceptual design phase of multirotor UAVs. The framework utilizes information from a comprehensive database of commercial lightweight multirotor UAVs. The database contains an extensive collection of crucial sizing parameters, performance metrics, and features associated with foldability and indoor guidance (e.g., obstacle avoidance sensors). These attributes specifically pertain to multirotor UAVs weighing less than 2kg, which exhibit diverse design and performance characteristics. The proposed ML framework employs multiple regression models (e.g. k-nearest neighbors regression, multi-layer perceptron regression) to predict the sizing parameters during a multirotor UAV’s conceptual design phase. This enables designers to make quick informed decisions, while also significantly reducing computational time and effort. Finally, the ML framework’s predictive capability is validated by comparing the predicted values with real-world data from an “unseen” test dataset.

Leveraging Machine Learning for Fraudulent Social Media Profile Detection

Abstract Fake social media profiles are responsible for various cyber-attacks, spreading fake news, identity theft, business and payment fraud, abuse, and more. This paper aims to explore the potential of Machine Learning in detecting fake social media profiles by employing various Machine Learning algorithms, including the Dummy Classifier, Support Vector Classifier (SVC), Support Vector Classifier (SVC) kernels, Random Forest classifier, Random Forest Regressor, Decision Tree Classifier, Decision Tree Regressor, MultiLayer Perceptron classifier (MLP), MultiLayer Perceptron (MLP) Regressor, Naïve Bayes classifier, and Logistic Regression. For a comprehensive evaluation of the performance and accuracy of different models in detecting fake social media profiles, it is essential to consider confusion matrices, sampling techniques, and various metric calculations. Additionally, incorporating extended computations such as root mean squared error, mean absolute error, mean squared error and cross-validation accuracy can further enhance the overall performance of the models.

Machine Learning Accelerated Discovery of Functional MXenes with Giant Piezoelectric Coefficients.

Efficient and rapid screening of target materials in a vast material space remains a significant challenge in the field of materials science. In this study, first-principles calculations and machine learning algorithms are performed to search for high out-of-plane piezoelectric stress coefficient materials in the MXene functional database among the 1757 groups of noncentrosymmetric MXenes with nonzero band gaps, which meet the criteria for piezoelectric properties. For the monatomic MXene testing set, the random forest regression (RFR), gradient boosting regression (GBR), support vector regression (SVR), and multilayer perceptron regression (MLPR) exhibit R2 values of 0.80, 0.80, 0.89, and 0.87, respectively. Expanding our analysis to the entire MXene data set, the best active learning cycle finds more than 140 and 22 MXenes with out-of-plane piezoelectric stress coefficients (e31) exceeding 3 × 10-10 and 5 × 10-10 C/m, respectively. Moreover, thermodynamic stabilities were confirmed in 22 MXenes with giant piezoelectric stress coefficients and 9 MXenes with both large in-plane (d11 > 15 pm/V) and out-of-plane (d31 > 2 pm/V) piezoelectric strain coefficients. These findings highlight the remarkable capabilities of machine learning and its optimization algorithms in accelerating the discovery of novel piezoelectric materials, and MXene materials emerge as highly promising candidates for piezoelectric materials.

Using Artificial Intelligence for Stock Price Prediction and Profitable Trading

Investing in the stock market is generally considered a safe and reliable way to protect and grow your savings over the long term. However, stock investments are risky and investing in the wrong companies can lead to financial loss. This research paper aims to see how the emerging field of Artificial Intelligence (AI) can be used to predict a stock’s future movement and capitalize on it by anticipating the movement and automatically buying or selling the stock, in an attempt to safely generate profit while minimizing risk. In this paper, a Multi-Layer Perceptron regressor model was trained to predict a stock’s future price using the stock’s previous prices, two industry competitors’ previous stock prices, and previous daily crude oil prices. The model was evaluated based on its average accuracy to the actual price in percent, and it was found that the model had an average 1.7% error on AAPL (Apple Inc) stock. An automated trading bot was then created and used the model’s daily output to perform a simulated backtest, where it was given a simulated $100,000 portfolio. Its performance was measured in multiple different trading scenarios. It was found that the model can reliably outperform stocks in fairly flat or downward market conditions and still profit on upward-moving stocks, but can still end up failing to profit if the underlying stock it was given drops in price significantly over the testing period.

Trajectory Predictor and Conflict Detection Figures of Merit for a Performance-Based Adaptive Air Traffic Monitoring System

This paper investigates the impact of trajectory predictor performance on the encounter probability generated by an adaptive conflict detection tool and examines the flexibility of the tool dependent on its adjustable thresholds, using historical radar track data. To achieve these objectives, two figures of merit were proposed: System Dynamic Range and System Tuning Envelope. To examine the conflict detection’s performance variability under different uncertainty levels and predictor types, simple multi-horizon trajectory predictors trained with two machine learning techniques of different characteristics are assessed at various look-ahead times: extreme gradient boosting with a discrete nature and a multi-layer perceptron regressor with a continuous nature. The results highlight the interdependence between the performances of the trajectory predictor and the conflict detector, and the quantification of this relationship can be represented through a sigmoid function. In addition, the two proposed figures of merit are effective for selecting suitable operating points in an adaptive conflict detector, based on dynamic thresholds and the performance requirements necessary for the trajectory predictors to achieve the expected detection performance at different look-ahead time.

Deep learning and big data mining for Metal–Organic frameworks with high performance for simultaneous desulfurization and carbon capture

Carbon capture and desulfurization of flue gases are crucial for the achievement of carbon neutrality and sustainable development. In this work, the “one-step” adsorption technology with high-performance metal–organic frameworks (MOFs) was proposed to simultaneously capture the SO2 and CO2. Four machine learning algorithms were used to predict the performance indicators (NCO2+SO2, SCO2+SO2/N2, and TSN) of MOFs, with Multi-Layer Perceptron Regression (MLPR) showing better performance (R2 = 0.93). To address sparse data of MOF chemical descriptors, we introduced the Deep Factorization Machines (DeepFM) model, outperforming MLPR with a higher R2 of 0.95. Then, sensitivity analysis was employed to find that the adsorption heat and porosity were the key factors for SO2 and CO2 capture performance of MOF, while the influence of open alkali metal sites also stood out. Furthermore, we established a kinetic model to batch simulate the breakthrough curves of TOP 1000 MOFs to investigate their dynamic adsorption separation performance for SO2/CO2/N2. The TOP 20 MOFs screened by the dynamic performance highly overlap with those screened by the static performance, with 76 % containing open alkali metal sites. This integrated approach of computational screening, machine learning, and dynamic analysis significantly advances the development of efficient MOF adsorbents for flue gas treatment.

Improvement of Machine Learning-Based Modelling of Container Ship’s Main Particulars with Synthetic Data

One of the main problems in the application of machine learning techniques is the need for large amounts of data necessary to obtain a well-generalizing model. This is exacerbated for studies in which it is not possible to access large amounts of data—for example, in the case of ship main data modelling, where a limited amount of real-world data (ship main data) is available for dataset creation. In this paper, a synthetic data generation technique has been applied to generate a large amount of synthetic data points regarding container ships’ main particulars. Models are trained using a multilayer perceptron (MLP) regressor on both original and synthetic data mixed with original data points. Then, the authors validate the performance of the obtained models on the original data and conclude whether a synthetic-data-based approach can be used to develop models in instances where the amount of data on ship main particulars may be limited. The results demonstrate an improvement across almost all outputs, ranging between 0.01 and 0.21 when evaluated using the coefficient of determination (R2) and between 0.27% and 3.43% when models are evaluated with mean absolute percentage error (MAPE). This indicates that the application of synthetic data can indeed be used for the improvement of ML-based model performance. The presented study demonstrates that the application of ML-based syncretization techniques can provide significant improvements to the process of ML-based determination of a ship’s main particulars at the early design stage. This paper suggests that, in cases where only a small dataset is available, artificial neural networks (ANN) can still be effectively employed to derive early-stage design values for the main particulars through the use of synthetic data.

Prediction of the water level at the Kien Giang River based on regression techniques

AbstractModel accuracy and runtime are two key issues for flood warnings in rivers. Traditional hydrodynamic models, which have a rigorous physical mechanism for flood routine, have been widely adopted for water level prediction in river, lake, and urban areas. However, these models require various types of data, in‐depth domain knowledge, experience with modeling, and intensive computational time, which hinders short‐term or real‐time prediction. In this paper, we propose a new framework based on machine learning methods to alleviate the aforementioned limitation. We develop a wide range of machine learning models such as linear regression (LR), support vector regression (SVR), random forest regression (RFR), multilayer perceptron regression (MLPR), and light gradient boosting machine regression (LGBMR) to predict the hourly water level at Le Thuy and Kien Giang stations of the Kien Giang river based on collected data of 2010, 2012, and 2020. Four evaluation metrics, that is, R2, Nash–Sutcliffe efficiency, mean absolute error, and root mean square error, are employed to examine the reliability of the proposed models. The results show that the LR model outperforms the SVR, RFR, MLPR, and LGBMR models.

Optimizing Neural Network Efficiency by Neuron Pruning

This study primarily focuses on two main objectives. The primary objective is to develop an appropriate model for accurately predicting the missing shear wave data in the Volve Oil Field located in the North Sea. By employing the Multi-Layer Perceptron Regression and modifying the neural network structure, the first goal attains a prediction accuracy of 0.943 for missing S-Wave log data. Furthermore, the objective of the study is to enhance the precision of forecasting incomplete S-Wave log data by optimising the structure of the artificial neural network, using neuron pruning techniques based on sensitivity analysis. This optimisation leads to a heightened accuracy rate of 0.9609. The effectiveness of these pruning strategies is clearly evident in their demonstrated capacity to improve the accurate prediction of missing data in the sonic wave log.

Analysis of heat transfer and prediction of preheating time of snow-melting on a highway T-beam bridge deck

Electrical heating is an innovative method of preventing icing of bridge decks during winter. Turning on the heating system before snowfall to prevent the icing of the bridge deck is more advantageous than melting already-formed ice. This article analyzes the heat-transfer performance of a bridge deck with heating cables during preheating before snowfall and predicts the preheating time of the bridge deck. Firstly, the temperature rise law and the state of the bridge deck with heating cables were studied through field experiments. Subsequently, a numerical model was developed. The effects of four factors on the heat-transfer performance of the bridge deck were analyzed. Finally, a bridge deck preheating-time model was established based on a multilayer perceptron regressor. This study enriches the application of neural networks and numerical simulations in preventing ice formation using the electrical heating method and is important for optimizing the heating system scheme and saving energy.

A Thermo-Structural Analysis of Die-Sinking Electrical Discharge Machining (EDM) of a Haynes-25 Super Alloy Using Deep-Learning-Based Methodologies

The most effective and cutting-edge method for achieving a 0.004 mm precision on a typical material is to employ die-sinking electrical discharge machining (EDM). The material removal rate (MRR), tool wear rate (TWR), residual stresses, and crater depth were analyzed in the current study in an effort to increase the productivity and comprehension of the die-sinking EDM process. A parametric design was employed to construct a two-dimensional model, and the accuracy of the findings was verified by comparing them to prior research. Experiments were conducted utilizing the EDM machine, and the outcomes were assessed in relation to numerical simulations of the MRR and TWR. A significant temperature disparity that arises among different sections of the workpiece may result in the formation of residual strains throughout. As a consequence, a structural model was developed in order to examine the impacts of various stress responses. The primary innovations of this paper are its parametric investigation of residual stresses and its use of Haynes 25, a workpiece material that has received limited attention despite its numerous benefits and variety of applications. In order to accurately forecast the output parameters, a deep neural network model, more precisely, a multilayer perceptron (MLP) regressor, was utilized. In order to improve the precision of the outcomes and guarantee stability during convergence, the L-BFGS solver, an adaptive learning rate, and the Rectified Linear Unit (ReLU) activation function were integrated. Extensive parametric studies allowed us to determine the connection between key inputs, including the discharge current, voltage, and spark-on time, and the output parameters, namely, the MRR, TWR, and crater depth.

Single Atom Alloys Segregation in the Presence of Ligands.

Single atom alloys (SAAs) have gained remarkable attention due to their tunable properties leading to enhanced catalytic performance, such as high activity and selectivity. The stability of SAAs is dictated by surface segregation, which can be affected by the presence of surface adsorbates. Research efforts have primarily focused on the effect of commonly found catalytic reaction intermediates, such as CO and H, on the stability of SAAs. However, there is a knowledge gap in understanding the effect of ligands from colloidal nanoparticle (NP) synthesis on surface segregation. Herein, we combine density functional theory (DFT) and machine learning to investigate the effect of thiol and amine ligands on the stability of colloidal SAAs. DFT calculations revealed rich segregation energy (Eseg) data of SAAs with d8 (Pt, Pd, Ni) and d9 (Ag, Au, Cu) metals exposing (111) and (100) surfaces, in the presence and absence of ligands. Using these data, we developed an accurate four-feature neural network using a multilayer perceptron regression (NN MLP) model. The model captures the underlying physics behind surface segregation in the presence of adsorbed ligands by incorporating features representing the thermodynamic stability of metals through the bulk cohesive energy, structural effects using the coordination number of the dopant and the ligands, the binding strength of the adsorbate to the metals, strain effects using the Wigner-Seitz radius, and electronic effects through electron affinities. We found that the presence of ligands makes the thermodynamics of segregation milder compared to the bare (nonligated) SAA surfaces. Importantly, the adsorption configuration (e.g., top vs bridge) and the binding strength of the ligand to the SAA surface (e.g., amines vs thiols) play an important role in altering the Eseg trends compared to the bare surface. We also developed an accurate NN MLP model that predicts Eseg in the presence of ligands to find thermodynamically stable SAAs, leading to the rapid and efficient screening of colloidal SAAs. Our model captures several experimental observations and elucidates complex physics governing segregation at nanoscale interfaces.

Impurity-Based Important Features for feature selection in Recursive Feature Elimination for Stock Price Forecasting

Stock investors perform stock price forecasting based on technical indicators and historical stock prices. The large number of technical indicators and historical data often leads to overfitting and ambiguity in forecasting using machine learning. In this paper, we proposed a feature selection approach using impurity-based important features in recursive feature elimination for stock price forecasting. The data utilized includes historical data and various moving averages. Feature selection is employed to reduce the number of features and obtain important and relevant features. The recursive feature elimination with impurity-based important features is utilized as the feature selection method. The machine learning methods employed are linear regression, support vector regression, multi-layer perceptron regression, and random forest regression. The measurement results of mean squared error (mse), root mean squared error (rmse), mean absolute error (mae), and mean absolute percentage error (mape) show that the optimal feature selection and machine learning method is achieved with six features and linear regression. The average mse, rmse, mae, and mape values are 0.000279, 0.016577, 0.012843, and 1.42236%, respectively. These results validate the effectiveness of impurity-based important features for feature selection in recursive feature elimination using historical data and various moving averages in stock price forecasting.

Strategic planning for cancer control: Utilizing machine-learning models to predict future incidences

One of the top health issues in the world is the Cancer. The number of patients with various types of cancer registered in clinics, oncology centers and hospitals are annually increasing. Predicting the estimated number of future cancer incidences for subsequent years is an important topic and needs further study. In this paper, officially registered cases at the Ministry of Health, Iraq and Oncology Center at Kirkuk governorate are used to forecast three years ahead estimated number of incidences for different types of cancer using four machine-learning models. The considered models are; gaussian processes, multilayer perceptron (MLP), MLP regressor, and sequential minimal optimization regressor (SMOreg). The study reveals significant differences among different models based on performance metrics. Based on SMOreg analysis in 2023, most types of cancer are expected to see an increase in cases. For instance, bladder cancer is projected to rise from 13 cases in 2020 to around 33 cases in 2023, a 153.8% increase. Results show that, the prediction of incidence cases using SMOreg model is outperformed other algorithms with minimized error in most types of cancer. According to the results, there will be a rise in the number of incidences in the 2023 for most types of cancer except breast and non-hodgkin lymphoma cancer, which expected a decrease in the number of cases. By utilizing of three-year-ahead annual numbers of cancer cases predictions, governments will be able to hedge financial risks, provide patients care and plan for cancer control programs.

Ensemble of optimised machine learning algorithms for predicting surface soil moisture content at a global scale

Abstract. Accurate information on surface soil moisture (SSM) content at a global scale under different climatic conditions is important for hydrological and climatological applications. Machine-learning-based systematic integration of in situ hydrological measurements, complex environmental and climate data, and satellite observation facilitate the generation of reliable data products to monitor and analyse the exchange of water, energy, and carbon in the Earth system at a proper space–time resolution. This study investigates the estimation of daily SSM using 8 optimised machine learning (ML) algorithms and 10 ensemble models (constructed via model bootstrap aggregating techniques and five-fold cross-validation). The algorithmic implementations were trained and tested using International Soil Moisture Network (ISMN) data collected from 1722 stations distributed across the world. The result showed that the K-neighbours Regressor (KNR) had the lowest root-mean-square error (0.0379 cm3 cm−3) on the “test_random” set (for testing the performance of randomly split data during training), the Random Forest Regressor (RFR) had the lowest RMSE (0.0599 cm3 cm−3) on the “test_temporal” set (for testing the performance on the period that was not used in training), and AdaBoost (AB) had the lowest RMSE (0.0786 cm3 cm−3) on the “test_independent-stations” set (for testing the performance on the stations that were not used in training). Independent evaluation on novel stations across different climate zones was conducted. For the optimised ML algorithms, the median RMSE values were below 0.1 cm3 cm−3. GradientBoosting (GB), Multi-layer Perceptron Regressor (MLPR), Stochastic Gradient Descent Regressor (SGDR), and RFR achieved a median r score of 0.6 in 12, 11, 9, and 9 climate zones, respectively, out of 15 climate zones. The performance of ensemble models improved significantly, with the median RMSE value below 0.075 cm3 cm−3 for all climate zones. All voting regressors achieved r scores of above 0.6 in 13 climate zones; BSh (hot semi-arid climate) and BWh (hot desert climate) were the exceptions because of the sparse distribution of training stations. The metric evaluation showed that ensemble models can improve the performance of single ML algorithms and achieve more stable results. Based on the results computed for three different test sets, the ensemble model with KNR, RFR and Extreme Gradient Boosting (XB) performed the best. Overall, our investigation shows that ensemble machine learning algorithms have a greater capability with respect to predicting SSM compared with the optimised or base ML algorithms; this indicates their huge potential applicability in estimating water cycle budgets, managing irrigation, and predicting crop yields.