Machine Learning Regression Research Articles

Comparative Analysis of Ensemble Learning Techniques and ANN for Prediction of Compressive Strength of Concrete Mix

Abstract: In this study, non-linear machine learning regressors and an artificial neural network (ANN) were modeled to predict the compressive strength of concrete mixes. A dataset of 1030 mix design data points with a wide range of compressive strength varying from M10 to M80 is taken into account. The data set comprised of 8 input parameters namely- cement, water, fine & coarse aggregate, flyash, slag, superplasticizer, and age. The paper also analyzes effects on the prediction of compressive strength due to waiving of the parameters with lesser importance factors. To improve the performances of models by reducing the variance, biases, and errors, K-fold cross-validation was examined. To examine the performances of the models, metrics such as coefficient of determination (R2) and mean square error (MSE) were employed. Most of the models can predict the compressive strength with good accuracy. Out of the 7-machine learning regressors and ANN models employed, Gradient Boosting Regressor yielded the best prediction accuracy of 0.9561 with the lowest MSE value of 0.0513.

Towards A Smart City Concept - Machine Learning Smarts for the Estimation of Future Temperature Rise in Tabuk City

In the Middle Eastern peninsula especially in Saudi Arabia, there is a varsity temperature variation among the individual regions. As far as the city of Tabuk is concerned, no study has been conducted, regarding climate change (the temperature rise) in the Tabuk region and its implications on society and for the flagship “Future Smart Cities” concept. In this paper, machine learning algorithms are used to predict the future temperature values in the Tabuk region. The machine learning algorithms were trained on the data collected from the real-time weather radar stations of the region. Different features from the dataset are used for machine learning models to predict the future temperature. These unique features, for example, humidity and pressure, impact the accurate predictability of the temperature. Temperature prediction is modelled as a regression problem due to the nature of the data, therefore, different machine learning regression models were developed, i.e. Artificial Neural Network ANN-based techniques (Multi-layer Perceptron (MLP)), Decision Trees (DT), K-Nearest Neighbours (KNN), and Support Vector Regression (SVR). Encouragingly, the preliminary model evaluations utilizing Mean Absolute Error (MAE) yielded a high accuracy of 90% on the testing dataset. The findings are envisaged to inform decision-makers in the Climate and Weather Ministry of Tabuk City, potentially contributing to the advancement of the city's "Future Smart Cities" concept.

Large-scale optimization of nonconvex MINLP refinery scheduling

Modeling and optimization of large-scale refinery scheduling problems is challenging because of their complexity and size. Herein, we propose a mathematical model to represent such problems more accurately and realistically, and a state-of-the-art optimization framework for its solution. The framework leverages the use of mathematical optimization and algorithmic methods by combining modeling approaches (process design, model decompositions), solving strategies (rescheduling, heuristics), and machine learning regression (surrogate models). An industrial-size refinery scheduling problem (2 blenders, 4 feed tanks, distillation network with 5 towers, processing network with FCC, hydrotreaters, debutanizers, superfractionator, catalytic reformer) is formulated as a hierarchical nonconvex mixed-integer nonlinear programming (MINLP) model and is successfully optimized, providing higher profitability and more efficient scheduling operations considering 12 feedstocks, 10 products and multiple scenarios for time horizon and step. Results highlight the importance of tuning scheduling parameters and employing an enhanced computer-aided framework to enable the solution of industrial refinery scheduling operations.

The search for sparse data in molecular datasets: Application of active learning to identify extremely low volatile organic compounds

The formation of aerosol particles in the atmosphere is driven by the gas to particle conversion of extremely low volatile organic compounds (ELVOC), organic compounds with a particularly low saturation vapor pressure (pSat). Identifying ELVOCs and their chemical structures is both experimentally and theoretically challenging: Measuring the very low pSat of ELVOCs is extremely difficult, and computing pSat for these often large molecules is computationally costly. Moreover, ELVOCs are underrepresented in available datasets of atmospheric organic species, which reduces the value of statistical models built on such data. We propose an active learning (AL) approach to efficiently identify ELVOCs in a data pool of atmospheric organic species with initially unknown pSat. We assess the performance of our AL approach by comparing it to traditional machine learning regression methods, as well as ELVOC classification based on molecular properties. AL proves to be a highly efficient method for ELVOC identification with limitations on the type of ELVOC it can identify. We also show that traditional machine learning or molecular property-based methods can be adequate tools depending on the available data and desired degree of efficiency.

Toward an improved ensemble of multi-source daily precipitation via joint machine learning classification and regression

Accurate estimation of precipitation at local to global scales can considerably enhance our understanding of climate system dynamics. While numerous precipitation products are available as indispensable tools for investigating precipitation and its associated processes, none can consistently provide the lowest estimation error across environmental conditions. The multiple source precipitation ensemble (MSPE) methods have been considered a vital solution. A new MSPE framework is proposed here, which simultaneously uses machine learning (ML) classification and regression techniques within an automatic workflow (MSPEaml). Six precipitation products and their ensembles based on different MSPE strategies were evaluated at 2365 gauged and 800 randomly selected ungauged sites over China. Results revealed significant precision inconsistencies among the products primarily due to their different data sources and retrieval algorithms; while MSPEaml can effectively reduce the random and classification errors of estimated precipitation according to the Kling-Gupta efficiency and Heidke skill score. The improvements demonstrated the unique features of MSPEaml, particularly the necessity of the joint use of ML classifiers and regressors and assigning spatiotemporal dynamic weights for merging precipitation data. Moreover, MSPEaml can substantially improve its generalizability through a simple binning procedure, making it applicable under more complex conditions. The varying contributions of predictor variables (indicated by Shapely values) in different ML models identified the complexity of the MSPE issue and further the importance of designing proper ML models according to specific targets. The proposed MSPE framework is expected to be a suitable solution for assembling multiple precipitation data sources with different time periods and scales.

Active-learning for global optimization of Ni-Ceria nanoparticles: The case of Ce4-xNixO8- x (x = 1, 2, 3).

Ni-CeO2 nanoparticles (NPs) are promising nanocatalysts for water splitting and water gas shift reactions due to the ability of ceria to temporarily donate oxygen to the catalytic reaction and accept oxygen after the reaction is completed. Therefore, elucidating how different properties of the Ni-Ceria NPs relate to the activity and selectivity of the catalytic reaction, is of crucial importance for the development of novel catalysts. In this work the active learning (AL) method based on machine learning regression and its uncertainty is used for the global optimization of Ce(4-x)NixO(8-x) (x = 1, 2, 3) nanoparticles, employing density functional theory calculations. Additionally, further investigation of the NPs by mass-scaled parallel-tempering Born-Oppenheimer molecular dynamics resulted in the same putative global minimum structures found by AL, demonstrating the robustness of our AL search to learn from small datasets and assist in the global optimization of complex electronic structure systems.

A comparative study of machine learning regression models for production systems condition monitoring

A comparative study of machine learning regression models for production systems condition monitoring

A multimodal stacked ensemble model for cardiac output prediction utilizing cardiorespiratory interactions during general anesthesia

This study examined the possibility of estimating cardiac output (CO) using a multimodal stacking model that utilizes cardiopulmonary interactions during general anesthesia and outlined a retrospective application of machine learning regression model to a pre-collected dataset. The data of 469 adult patients (obtained from VitalDB) with normal pulmonary function tests who underwent general anesthesia were analyzed. The hemodynamic data in this study included non-invasive blood pressure, plethysmographic heart rate, and SpO2. CO was recorded using Vigileo and EV1000 (pulse contour technique devices). Respiratory data included mechanical ventilation parameters and end-tidal CO2 levels. A generalized linear regression model was used as the metalearner for the multimodal stacking ensemble method. Random forest, generalized linear regression, gradient boosting machine, and XGBoost were used as base learners. A Bland–Altman plot revealed that the multimodal stacked ensemble model for CO prediction from 327 patients had a bias of − 0.001 L/min and − 0.271% when calculating the percentage of difference using the EV1000 device. Agreement of model CO prediction and measured Vigileo CO in 142 patients reported a bias of − 0.01 and − 0.333%. Overall, this model predicts CO compared to data obtained by the pulse contour technique CO monitors with good agreement.

Machine learning-based prediction and experimental validation of heavy metal adsorption capacity of bentonite

As a natural adsorbent material, bentonite is widely used in the field of heavy metal adsorption. The heavy metal adsorption capacity of bentonite varies significantly in studies due to the differences in the properties of bentonite, solution, and heavy metal. To achieve accurate predictions of bentonite's heavy metal adsorption capacity, this study employed six machine learning (ML) regression algorithms to investigate the adsorption characteristics of bentonite. Finally, an eXtreme Gradient Boosting Regression (XGB) model with outstanding predictive performance was constructed. Explanation analysis of the XGB model further reveal the importance and influence manner of each input feature in predicting the heavy metal adsorption capacity of bentonite. The feature categories influencing heavy metal adsorption capacity were ranked in order of importance as adsorption conditions > bentonite properties > heavy metal properties. Furthermore, a web-based graphical user interface (GUI) software was developed, facilitating researchers and engineers to conveniently use the XGB model for predicting the heavy metal adsorption capacity of bentonite. This study provides new insights into the adsorption behaviors of bentonite for heavy metals, offering guidance and support for enhancing its application efficiency and addressing heavy metal pollution remediation.

Assessment and regression of carbon emissions from the building and construction sector in China: A provincial study using machine learning

The building and construction sector is a major contributor to carbon emissions in China. Hence, it is crucial to explore the characteristics and trends of building carbon emissions to achieve the carbon peak and neutrality. While previous studies have made efforts to analyze the influencing factors through different approaches, developing an effective and intelligent regression model based on machine learning algorithms remains challenging in predicting the carbon emission trend. This study analyzed carbon emissions and per capita indicators of the building and construction sector in 30 provincial regions in China from 2005 to 2021. While embodied and operational carbon emissions contribute equally to the total emissions, the results showed a significant spatial-temporal correlation. Considering the emissions as target features, nine alternative machine learning regression models were developed using eight identified explanatory features incorporating scale, economic, technological, and classification factors. Based on performance metrics encompassing root mean squared error, coefficient of determination, and mean absolute percentage error, the stacking ensemble regression model was identified to have superior performance. This model was further employed to conduct a sensitivity analysis of explanatory features on carbon emissions. The results indicated that urbanization rate and population were the most sensitive factors, with varying effects on different target features. These findings can be used to predict carbon emission trends and promote carbon reduction policies in the building industry.

A low-cost machine learning process for gait measurement using an electrostatic sensors network

Continuous in-house measurement of gait of elderly People is relevant for health professionals. To be adopted by most, the system must be low-cost and non-intrusive. In this paper we present a solution for measuring the walking velocity based on a network of 4 electric potential sensors. In our experiments, we also add PIR sensors used in our previous work for comparative purposes. A temporary Depth camera is used for training the model on walking velocity. The first results presented are obtained without machine learning. Then a machine learning regression method is tested to reduce the uncertainty of the sensors. The results show that the electric potential sensors are suitable for the in-house measurement of walking speed of elderly people. The uncertainty is lower than the target of 0.15 m s-1 known as the upper limit for detecting a reduction in speed due to illness. As for the PIR sensors, electric potential sensors consume very little energy, they are inexpensive, they can be embedded and hidden in the home which makes them less -intrusive and furthermore have better accuracy.

Comparative Analysis Using Multiple Regression Models for Forecasting Photovoltaic Power Generation

Effective machine learning regression models are useful toolsets for managing and planning energy in PV grid-connected systems. Machine learning regression models, however, have been crucial in the analysis, forecasting, and prediction of numerous parameters that support the efficient management of the production and distribution of green energy. This article proposes multiple regression models for power prediction using the Sharda University PV dataset (2022 Edition). The proposed regression model is inspired by a unique data pre-processing technique for forecasting PV power generation. Performance metrics, namely mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), R2-score, and predicted vs. actual value plots, have been used to compare the performance of the different regression. Simulation results show that the multilayer perceptron regressor outperforms the other algorithms, with an RMSE of 17.870 and an R2 score of 0.9377. Feature importance analysis has been performed to determine the most significant features that influence PV power generation.

Machine learning regression algorithms for generating chemical element maps from X-ray fluorescence data of paintings

Generating chemical element maps of paintings from X-ray fluorescence (XRF) data is a very valuable tool for the scientific community of conservators and art historians. Hand-held XRF scanners are cheap and easily portable but their use provides scans with a few data, so additional analytical tools are needed to obtain reliable chemical element maps from them. Recently, the software tool SmART_Scan was released, which uses an algorithm based on the minimum hypercube distance (MHD) to compute this kind of maps. In this paper, we propose a new methodology to address this problem by using machine learning algorithms for regression as alternative and more accurate techniques than MHD. We tested MHD versus eight machine learning regression algorithms on two paintings with different features. Our results showed that machine learning algorithms Random Forest and kNN significantly outperformed MHD in Mean Squared Error (MSE) and coefficient of determination (R2) for all the experiments. When using experts’ data and a hold-out validation, kNN was the best-ranked algorithm. Random Forest was the best-ranked algorithm when cross-validation was used. We did not find significant differences in average MSE nor in R2 between kNN and Random Forest, so we can conclude that Random Forest is the best-suited algorithm for computing chemical element maps of paintings from XRF data.

Data driven models for capacity prediction of CFS lipped channel flexural members

Cold-Formed Steel (CFS) is considered as a sustainable material and CFS members are becoming popular nowadays as primary or secondary load carrying structural elements. The increasing use of CFS members is owing to their other advantages like high strength to weight ratio, versatility in cross sectional shapes and dimensions, easy fabrication and mass production, uniform quality, good energy efficiency, easiness to handle and transport etc. The behaviour of cold-formed members under flexure is complex as it involves the occurrence of various buckling modes (Local, distortional or lateral-torsional buckling) and the possible interaction between them. Although, the moment carrying capacity of CFS beams can be estimated using existing codes and design standards, the present study is an attempt to introduce an alternate rapid capacity assessment tool based on machine learning for the same. Available data on experiments conducted on CFS lipped channel beams are collected to model and analyse in ABAQUS for validating the finite element model. The validated FE model is used for developing the input-output data set for training the Machine Learning (ML) models by sampling the geometric and strength related input parameters of the CFS beam. Recently introduced machine learning regression models like Linear Regression, Lasso regression, K-Nearest Neighbors, Decision Tree, Random Forest, Adaptive Boosting, Extreme Gradient Boosting, Light Gradient Boosting, Categorical Boosting, Gradient Boosting Regressor, Support Vector Machine and Artificial Neural Network are used to predict the ultimate flexural capacity of CFS beams. The CatBoost model is emerged as the best performing algorithm for predicting the flexural capacity, with 98.5% accuracy for the test data set. A SHAP (SHapley Additive exPlanations) analysis is also conducted for the global and local interpretation of the prediction of the best performing machine learning model. SHAP results not only indicate the influence of input parameters on output prediction, but also the influence of input parameters on each other in predicting the output.

Prediction on compressive strength of recycled aggregate self-compacting concrete by machine learning method

The compressive strength is generally a crucial mechanical indicator for evaluating the quality of recycled aggregate self-compacting concrete (RASCC). To obtain a reliable prediction result of the compressive strength of RASCC, machine learning methods, including artificial neural network (ANN), random tree (RT), bagging, and random forest (RF), are utilized to predict the compressive strength of RASCC in this study. To build predictive models, 18 features and 289 data samples were collected from previous literature. The prediction effects of 4 artificial intelligence (AI) models on compressive strength of RASCC are compared. Four statistical parameters were used to evaluate the performance of the models, and a comparison was made between the model-predicted results and the experimental results. A good correlation between machine learning models and experimental results was obtained. Sensitivity investigation reveals that the cement content and the apparent density of natural coarse aggregate have the greatest influence on compressive strength. The study demonstrates the potential of ANN and RF models as useful tools that can support mortar design and/or optimization, with higher accuracy and good interpretability. According to this study, machine learning regression techniques hold great potential as instruments for forecasting RASCC's compressive strength.

Continuous monitoring of grassland AGB during the growing season through integrated remote sensing: a hybrid inversion framework

ABSTRACT Inverting grassland above-ground biomass (AGB) presents a significant challenge due to difficulties in characterizing leaf physiological states and obtaining accurate ground-truth data. This study introduces an innovative hybrid model for AGB inversion based on the AGB = leaf mass per area (LMA) * leaf area index (LAI) paradigmn in the Ewenki Banner region of Inner Mongolia. The model integrates the PROSAIL radiative transfer model, machine learning regression, LEnKF data assimilation theory, multisource remote sensing, and meteorological data, following a four-step approach. Firstly, we establish LAI and LMA inversion models by combining the PROSAIL model with machine learning techniques. Secondly, data assimilation fuses the PROSAIL-derived LAI with MODIS-LAI. In the third phase, a Random Forest predictive model is developed for LMA estimation. Lastly, the accuracy of the hybrid model is assessed using empirical data. Precision evaluation with ground-truth samples demonstrates that the assimilated LAI and RF-predicted LMA yield the lowest prediction error for grassland AGB (RMSE = 0.0033 g/cm2; MAE = 0.0028 g/cm2). This model framework addresses the challenge of limited prior knowledge in the PROSAIL-AGB prediction model, thereby enhancing the prediction accuracy while maintaining its key advantages: providing continuous observations at high spatiotemporal resolutions without relying on measured sample data.

Identification of red blood cell distribution width as a prognostic factor in acute myeloid leukemia

Many prognostic factors have been identified in acute myeloid leukemia (AML). In this study, we investigated novel prognostic biomarkers using machine learning and Cox regression models in a prospective cohort of 591 patients with AML and tried to identify potential therapeutic targets based on transcriptomic data. We found that elevated red blood cell distribution width (RDW) at diagnosis was an adverse prognostic factor for AML, independent of the 2022 European LeukemiaNet (ELN2022) genetic risk. As a continuous variable, higher RDW was associated with shorter overall survival (OS) (hazard ratio [HR] 1.087, 95% confidence interval [CI] 1.036-1.139, p < 0.001) and event-free survival (EFS) (HR 1.078, 95% CI 1.033-1.124, p < 0.001). Elevated RDW returned to normal after consolidation therapy, which indicated that leukemia cells resulted in abnormal RDW. We further investigated the relationship between RDW and transcriptome in another cohort of 191 patients with AML and public datasets using gene set enrichment analysis (GSEA) and cell-type identification by estimating relative subsets of RNA transcripts (CIBERSORT). We found that patients in the high-RDW group were significantly enriched in the positive regulation of erythroid differentiation and inflammation-related pathways. Finally, we identified the inflammation-associated gene IL12RB2 and verified its prognostic relevance with patients with AML in public databases, suggesting it as a potential therapy target.

Soil liquefaction in seismic events: pioneering predictive models using machine learning and advanced regression techniques

Soil liquefaction in seismic events: pioneering predictive models using machine learning and advanced regression techniques

Cuffless blood pressure estimation from photoplethysmography using deep convolutional neural network and transfer learning

Blood pressure (BP) monitoring is an essential indicator for diseases of the cardiovascular system. Early detection of abnormalities in BP helps to significantly reduce the risk of diseases such as chronic heart failure, stroke and hypertension. In this study, we propose a new framework for systolic, diastolic and mean arterial blood pressure (SBP, DBP and MBP) estimation using PPG signals and its derivatives. The proposed framework includes the extraction of deep features by pre-trained CNN models based on transfer learning using images of signal waveforms. The most distinctive features are obtained by using the RFE feature selection algorithm on the extracted deep features. The selected deep features are exposed to a range of well-known machine and deep learning regression methods. The proposed BP estimation models have been evaluated with statistical metrics, visual analytical tools and international gold standards such as AAMI and BHS. Experimental results show that the first derivative of PPG, the VPG input image, the deep features obtained with DenseNet121 and the bi-directional gated recurrent unit (Bi-GRU) algorithm provide the best BP estimation performance. The results reveal that the SBP, DBP and MBP estimation models are grade-A according to the BHS protocol and meet the AAMI standard. The presented framework has been compared with well-established studies using the MIMIC II dataset, and the comparative analysis confirms that the proposed method outperforms existing techniques. The deep learning model for BP estimation offers a highly accurate, fast and practical system, especially in the monitoring of patients at risk of hypertension.

Estimating Acceleration from a Single Uniform Linear Motion-Blurred Image using Homomorphic Mapping and Machine Learning

Context: Vision-based measurement (VBM) systems are becoming popular as an affordable and suitable alternative for scientific and engineering applications. When cameras are used as instruments, motion blur usually emerges as a recurrent and undesirable image degradation, which in fact contains kinematic information that is usually dismissed. Method: This paper introduces an alternative approach to measure relative acceleration from a real invariant uniformly accelerated linear motion-blurred image. This is done by using homomorphic mapping to extract the characteristic Point Spread Function (PSF) of the blurred image, as well as machine learning regression. A total of 125 uniformly accelerated motion-blurred pictures were taken in a light- and distance-controlled environment, at five different accelerations ranging between 0,64 and 2,4 m/s2. This study evaluated 19 variants such as tree ensembles, Gaussian processes (GPR), and linear, support vector machine (SVM), and tree regression. Results: The best RMSE result corresponds to GPR (Matern 5/2), with 0,2547 m/s2 and a prediction speed of 530 observations per second (obs/s). Additionally, some novel deep learning methods were used to obtain the best RMSE value (0,4639 m/s2 for Inception ResNet v2, with a prediction speed of 11 obs/s. Conclusions: The proposed method (homomorphic mapping and machine learning) is a valid alternative for calculating acceleration from invariant motion blur in real-time applications when additive noise is not dominant, even surpassing the deep learning techniques evaluated.