Regression-based Machine Learning Models Research Articles

Semi-Analytical Model for Hybrid and Redox Targeting Flow Battery Systems

Redox flow battery represents an economically viable energy storage technology that can integrate intermittent renewable energies from solar and wind power into existing electric grids. Besides the conventional all-liquid redox flow battery, there are two different architectures of flow batteries: hybrid and redox targeting flow batteries. Because the reactive solid phase is involved in the system, the energy storage density increases significantly. Therefore, they are considered as ones of the potential solutions for the future large-scale deployment of long-duration energy storage systems (LDES). A digital twin (DT) has been demonstrated as a promising numerical framework and platform to assist in the design, deployment, and operation of LDES. DTs usually require a fast-response model to provide an estimation of the flow battery's performance, which can be a fast-forward physics-based model or a machine learning-based regression model. Many pioneering works and our previous studies have introduced several fast-forward physics-based models for the conventional all-liquid redox flow battery in both inorganic and organic systems. In this study, a semi-analytical model, which can adapt to DTs, is proposed to provide an estimation of the performance of the hybrid and the redox targeting flow battery at different operating conditions.

Semiconductors for enhanced solar photovoltaic-thermoelectric 4E performance optimization: Multi-objective genetic algorithm and machine learning approach

In this study, a groundbreaking exploration of a concentrated photovoltaic-thermoelectric (CPV-TE) module employing an unprecedented selection of six thermoelectric materials across diverse temperature ranges is presented. This work innovatively employs a multi-objective optimization framework that combines genetic algorithms and goal attainment methods, aiming to optimize energy, exergy, environmental, and economic (4 E) performance. Notably, this study is the first to construct and evaluate five regression-based machine learning models with an emphasis on minimal root mean squared error and mean absolute error for rapid CPV-TE performance predictions, essential for real-world applications. Our analysis unveils that among the tested materials, the CPV-TE module incorporating lead telluride (PbTe) achieves the highest 4 E performance, demonstrating a peak exergy efficiency of 14.2 %, and annual energy savings and carbon reduction of 13.5 kWh and 6.4 kg, respectively. Furthermore, the Gaussian Process Regression model is identified as the most effective among the machine learning models for forecasting performance across the materials. This study significantly advances the field by providing novel insights into thermoelectric material selection and optimization for CPV-TE modules, and establishing pioneering forecasting tools that catalyze the efficient deployment of these systems.

Performance Evaluation of Regression-Based Machine Learning Models for Modeling Reference Evapotranspiration with Temperature Data

Performance Evaluation of Regression-Based Machine Learning Models for Modeling Reference Evapotranspiration with Temperature Data

QUAS(AI)R: A Novel Machine Learning Algorithm to Predict X-ray Brightness in Active Galactic Nuclei

Active Galactic Nuclei (AGNs) are a compact region at the center of galaxies that emit more energy than the rest of the galaxy itself. They emit light across the electromagnetic spectrum, from radio waves to optical light to high-energy X-rays. AGNs indicate the existence of highly energetic phenomena in the nucleus of the galaxy. Although AGNs were identified 60 years ago, our knowledge about their physical properties is limited. Quasars, a subtype of AGNs, provide some of the most intense forms of X-rays, which are among the most energetic light known. Furthering our understanding of X-rays in the dynamic environments of quasars will add to our understanding of how to benefit from their use on Earth. In my project, I study the X-ray brightness in quasars to develop six types of regression-based machine learning models for the X-ray brightness predictions. These six models were Stochastic Gradient Descent (SGD), Random Forest, Ridge, Lasso, Bayesian and the baseline linear regression model, built on the scikit-learn Python package. The training/testing split on the MILLIQUAS dataset was 80/20 percent, and each model was tuned on model-specific hyperparameters. Benchmarked with the normalized mean absolute error (NMAE), the top three performing models were the Bayesian (0.022%), Ridge (0.180%), and Lasso (0.183%), with the baseline NMAE at 0.284%. With this, we can learn more about the evolution of galaxies in the early Universe and understand how these dynamic environments came to be.

Computation cost reduction in 3D shape optimization of nanophotonic components

Inverse design methodologies effectively optimize many design parameters of a photonic device with respect to a primary objective, uncovering locally optimal designs in a typically non-convex parameter space. Often, a variety of secondary objectives (performance metrics) also need to be considered before fabrication takes place. Hence, a large collection of optimized designs is useful, as their performance on secondary objectives often varies. For certain classes of components such as shape-optimized devices, the most efficient optimization approach is to begin with 2D optimization from random parameter initialization and then follow up with 3D re-optimization. Nevertheless, the latter stage is substantially time- and resource-intensive. Thus, obtaining a desired collection of optimized designs through repeated 3D optimizations is a computational challenge. To address this issue, a machine learning-based regression model is proposed to reduce the computation cost involved in the 3D optimization stage. The regression model correlates the 2D and 3D optimized structural parameters based on a small dataset. Using the predicted design parameters from this model as the initial condition for 3D optimization, the same optima are reached faster. The effectiveness of this approach is demonstrated in the shape optimization-based inverse design of TE0-TE1 mode converters, an important component in mode-division multiplexing applications. The final optimized designs are identical in both approaches, but leveraging a machine learning-based regression model offers a 35% reduction in computation load for the 3D optimization step. The approach provides a more effective means for sampling larger numbers of 3D optimized designs.

A comparative study of machine learning-based regression models for supply chain management

The rise of machine learning technology has opened up unprecedented opportunities for the retail industry. Machine learning, as an essential branch of artificial intelligence, enables computers to improve their performance through continuous learning and experience. It has demonstrated its ability to handle large-scale data and complex problems effectively. In retail, machine learning predictions and methods can also lead to significant breakthroughs in supply chain management, helping businesses identify better ways to maintain economic stability and growth, which are crucial for improving people's living standards, eliminating poverty, promoting social stability, driving technological progress, and reducing inequality. This is achieved through different algorithmic regression methods, which can predict future trends and consumer behavior with high accuracy. Machine learning algorithms can analyze vast amounts of data to identify patterns and trends and make accurate predictions about future demand, product inventory levels, and other important factors that drive business success in the retail industry.

Wind speed prediction for site selection and reliable operation of wind power plants in coastal regions using machine learning algorithm variants

The challenge of predicting wind speeds to facilitate site selection and the consistent operation of wind power plants in coastal regions is a global concern. The output of wind turbines is subject to fluctuations corresponding to changes in wind speed. The unpredictable characteristics of wind patterns introduce vulnerabilities to wind power facilities in wind power plants. To address this unpredictability, an effective strategy involves forecasting wind speeds at specific locations during wind power plant operations. While previous research has explored various machine learning algorithms to tackle these issues, satisfactory results have not been achieved, and Bangladesh faces challenges in this regard, especially in low-wind speed areas. This study aims to identify the most accurate machine learning-based algorithm to forecast the short-term wind speed of two areas (Kutubdia and Cox's Bazar) located on the eastern coast of Bangladesh. Wind speed data for a span of 21.5 years, ranging from January 2001 to June 2022, were sourced from two outlets: the Bangladesh Meteorological Department and the website of NASA. Wind speed has been forecasted using 14 different regression-based machine learning models with a comprehensive overview. The results of the experiment highlight the exceptional predictive performance of a boosting-based ensemble method known as categorical boosting, especially in the context of forecasting wind speed data obtained from NASA. Based on the testing data, the evaluation yields remarkable results, with coefficients of determination measuring 0.8621 and 0.8758 for wind speed in Kutubdia and Cox's Bazar, respectively. The study underscores the critical importance of prioritizing optimal turbine site selection in the context of wind power facilities in Bangladesh. This approach can yield benefits for stakeholders, including engineers and project owners associated with wind projects.

Risky behaviors and road safety: An exploration of age and gender influences on road accident rates.

Human behavior is a dominant factor in road accidents, contributing to more than 70% of such incidents. However, gathering detailed data on individual drivers' behavior is a significant challenge in the field of road safety. As a result, researchers often narrow the scope of their studies thus limiting the generalizability of their findings. Our study aims to address this issue by identifying demographic-related variables and their indirect effects on road accident frequency. The theoretical basis is set through existing literature linking demographics to risky driving behavior and through the concept of "close to home" effect, finding that the upwards of 62% of accidents happen within 11km of a driver's home. Using regression-based machine learning models, our study, looking at England, UK, explores the theoretical linkages between demographics of an area and road accident frequency, finding that census data is able to explain over 28% of the variance in road accident rates per capita. While not replacing more in-depth research on driver behavior, this research validates trends found in the literature through the use of widely available data with the use of novel methods. The results of this study support the use of demographic data from the national census that is obtainable at a large spatial and temporal scale to estimate road accident risks; additionally, it demonstrates a methodology to further explore potential indirect relationships and proxies between behaviors and road accident risk.

Regression based machine learning models for forecasting preterm birth cases

Preterm birth (PTB) is a significant health issue, leading to neonatal mortality and morbidity worldwide. However, effective forecasting model may assist to reduce the rate of PTB cases. In this study, three different regression-based forecasting models namely simple linear regression (SLR), multi-variate linear regression (MLR) and standard ARIMA are introduced over the data collected (during 01.01.2017-01.05.2022) from Bokaro hospital of India for medium-term forecasting of PTB cases. The models are trained and validated over parts of the dataset. On the basis of forecasting-accuracy (%age), RMSE and R-square values achieved over the validation-period, the proposed MLR model is the more accurate one to forecast PTB cases.

Foot Clearance Prediction using Wrist Acceleration and Gait Speed

Elderly individuals experience fall accidents due to tripping because recognizing foot clearance during walking is difficult for them. To prevent fall accidents, foot clearance should be measured and informed in daily life. Foot clearance is commonly measured using vision-based systems, such as optical motion capture systems. However, problem of these vision-based systems is that these systems cannot measure foot clearance in daily life because they have limitations due to obstacles and field of view. Based on this problem, we developed a wearable fall prevention system using smart devices, such as smartphones and smartwatches. This study aimed to evaluate the proposed prediction method for foot clearance using sensor data obtained from wearable smart devices which can be used in daily life. The proposed method will contribute to measure foot clearance in daily life. This method predicts foot clearance from wrist acceleration and gait speed using a machine learning-based regression model. The proposed method was tested in a computational simulation with a public gait dataset obtained using an optical motion capture system. The results showed that the correlations between the predicted and actual foot clearance were at least 0.65. In conclusion, this study indicates the possibility that the proposed method can be used to measure foot clearance and thus can be used in wearable fall prevention systems.

Development and performance comparison of optimized machine learning-based regression models for predicting energy-related carbon dioxide emissions.

Accurate prediction of CO2 emissions for the countries has become a crucial task in decision-making processes for planning energy conversion and usage, supporting the design of effective emissions reduction strategies, and helping to achieve the goal of a sustainable and low-carbon future. Therefore, this study aims to develop a general model that can predict the national CO2 emissions of each country using data from 68 countries with high prediction accuracy based on machine learning regression models. Nine prediction models were developed using Support Vector Regression, Ensemble of Trees, and Gaussian Process Regression algorithms as machine learning methods, and their prediction performances were compared. Additionally, the hyperparameters of these three machine-learning methods were tuned by Bayesian optimization to improve their prediction performance. The test results of the optimized Gaussian Process Regression model (MSE = 106.68, RMSE = 10.328, MAE = 4.904, MAPE = 3.38%, R2 = 0.9998) showed that it was the best prediction model among the all developed models. Additionally, the optimized Gaussian Process Regression model gave very robust results in predicting CO2 emissions in many countries, indicating that it can be used reliably and with high accuracy as a promising prediction model.

Early qualitative and quantitative amplitude-integrated electroencephalogram and raw electroencephalogram for predicting long-term neurodevelopmental outcomes in extremely preterm infants in the Netherlands: a 10-year cohort study

Extremely preterm infants (<28 weeks of gestation) are at great risk of long-term neurodevelopmental impairments. Early amplitude-integrated electroencephalogram (aEEG) accompanied by raw EEG traces (aEEG-EEG) has potential for predicting subsequent outcomes in preterm infants. We aimed to determine whether and which qualitative and quantitative aEEG-EEG features obtained within the first postnatal days predict neurodevelopmental outcomes in extremely preterm infants. This study retrospectively analysed a cohort of extremely preterm infants (born before 28 weeks and 0 days of gestation) who underwent continuous two-channel aEEG-EEG monitoring during their first 3 postnatal days at Wilhelmina Children's Hospital, Utrecht, the Netherlands, between June 1, 2008, and Sept 30, 2018. Only infants who did not have genetic or metabolic diseases or major congenital malformations were eligible for inclusion. Features were extracted from preprocessed aEEG-EEG signals, comprising qualitative parameters grouped in three types (background pattern, sleep-wake cycling, and seizure activity) and quantitative metrics grouped in four categories (spectral content, amplitude, connectivity, and discontinuity). Machine learning-based regression and classification models were used to evaluate the predictive value of the extracted aEEG-EEG features for 13 outcomes, including cognitive, motor, and behavioural problem outcomes, at 2-3 years and 5-7 years. Potential confounders (gestational age at birth, maternal education, illness severity, morphine cumulative dose, the presence of severe brain injury, and the administration of antiseizure, sedative, or anaesthetic medications) were controlled for in all prediction analyses. 369 infants were included and an extensive set of 339 aEEG-EEG features was extracted, comprising nine qualitative parameters and 330 quantitative metrics. The machine learning-based regression models showed significant but relatively weak predictive performance (ranging from r=0·13 to r=0·23) for nine of 13 outcomes. However, the machine learning-based classifiers exhibited acceptable performance in identifying infants with intellectual impairments from those with optimal outcomes at age 5-7 years, achieving balanced accuracies of 0·77 (95% CI 0·62-0·90; p=0·0020) for full-scale intelligence quotient score and 0·81 (0·65-0·96; p=0·0010) for verbal intelligence quotient score. Both classifiers maintained identical performance when solely using quantitative features, achieving balanced accuracies of 0·77 (95% CI 0·63-0·91; p=0·0030) for full-scale intelligence quotient score and 0·81 (0·65-0·96; p=0·0010) for verbal intelligence quotient score. These findings highlight the potential benefits of using early postnatal aEEG-EEG features to automatically recognise extremely preterm infants with poor outcomes, facilitating the development of an interpretable prognostic tool that aids in decision making and therapy planning. European Commission Horizon 2020.

Accurate estimation of sorghum crop water content under different water stress levels using machine learning and hyperspectral data.

This study investigates the effects of different water stress levels on spectral information, leaf area index (LAI), and the performance of three machine learning (ML) algorithms in estimating crop water content (CWC) of sorghum. The results show that the spectral reflectance of sorghum varies with growth stage and irrigation treatment, but consistent patterns are observed for each treatment. The LAI of sorghum gradually increased throughout the growth stages, with the most significant variation observed during the flowering stage. In this study, three machine learning-based regression models, namely, extreme gradient boosting (XGBoost), random forest (RF), and support vector machine (SVM), were utilized to estimate sorghum CWC using hyperspectral measurements. Recursive feature elimination (RFE) method was used to select the optimal spectral reflectance wavelengths for the ML models, and principal component analysis (PCA) was used to reduce the dimensionality of the hyperspectral data. The results indicated that the RF model achieved the highest R2 (0.90) and lowest of RMSE (56.05) value using selected wavelengths, while the XGBoost model demonstrated superior accuracy and reliability in estimating CWC using dimensionality-reduced hyperspectral data (r = 0.96, RMSE = 45.77). Also, the study highlights the importance of vegetation index (VI) in CWC estimate. Some VIs, such as NDVI and MSAVI, performed poorly, while others, such as CL_Rededge and EVI, performed better. The study provides valuable insights into the effects of water stress levels on spectral information, LAI, and the performance of ML algorithms in estimating the CWC of sorghum. The findings have significant implications for precision agriculture, as accurate and reliable estimates of CWC can help farmers optimize irrigation and fertilizer applications, leading to improved crop yields and resource efficiency.

Development of prediction software to describe total mesophilic bacteria in spinach using a machine learning-based regression approach.

The purpose of this study was to create a tool for predicting the growth of total mesophilic bacteria in spinach using machine learning-based regression models such as support vector regression, decision tree regression, and Gaussian process regression. The performance of these models was compared to traditionally used models (modified Gompertz, Baranyi, and Huang models) using statistical indices like the coefficient of determination (R2) and root mean square error (RMSE). The results showed that the machine learning-based regression models provided more accurate predictions with an R2 of at least 0.960 and an RMSE of at most 0.154, indicating that they can be used as an alternative to traditional approaches for predictive total mesophilic. Therefore, the developed software in this work has a significant potential to be used as an alternative simulation method to traditionally used approach in the predictive food microbiology field.

An In-Ear PPG-Based Blood Glucose Monitor: A Proof-of-Concept Study.

Monitoring diabetes saves lives. To this end, we introduce a novel, unobtrusive, and readily deployable in-ear device for the continuous and non-invasive measurement of blood glucose levels (BGLs). The device is equipped with a low-cost commercially available pulse oximeter whose infrared wavelength (880 nm) is used for the acquisition of photoplethysmography (PPG). For rigor, we considered a full range of diabetic conditions (non-diabetic, pre-diabetic, type I diabetic, and type II diabetic). Recordings spanned nine different days, starting in the morning while fasting, up to a minimum of a two-hour period after eating a carbohydrate-rich breakfast. The BGLs from PPG were estimated using a suite of regression-based machine learning models, which were trained on characteristic features of PPG cycles pertaining to high and low BGLs. The analysis shows that, as desired, an average of 82% of the BGLs estimated from PPG lie in region A of the Clarke error grid (CEG) plot, with 100% of the estimated BGLs in the clinically acceptable CEG regions A and B. These results demonstrate the potential of the ear canal as a site for non-invasive blood glucose monitoring.

Partial least square regression based machine learning models for soil organic carbon prediction using visible–near infrared spectroscopy

Monitoring and assessment of soil organic carbon (SOC) are critical for maintaining and enhancing the productivity of agricultural soils. The SOC is commonly determined through soil sampling and subsequent laboratory analysis using chemical methods. This method though very precise is time-consuming, labour-intensive and expensive. Contrarily, visible and near-infrared reflectance spectroscopy (VNIRS) may be utilised to estimate SOC in a quick, labour-saving, and cost-effective manner. In this study, 72 soil samples were collected for SOC estimation and spectra collection. This current work proposes to investigate the use of PLSR scores in place of raw spectral reflectance to increase both the computation and model efficiency by reducing the number of input variables while retaining the maximum information present in the original data. With the existing indices, ratio and normalized difference indices were calculated in all possible combinations and were regressed to SOC content to identify the best-performing indices. Ten different multivariate models were evaluated for SOC estimation using full-spectrum and partial least squares regression (PLSR) scores. The results revealed that reflectance gradually increased with increasing soil depth and decreasing SOC. The prediction models developed using existing indices were observed to be poor in predicting the SOC with the R2 values ranging from 0.009 to 0.34. The best spectral indices for estimating SOC were RI (R1888, R2015) and NDI (R1888, R2015) with R2 of 0.60, 0.61 and 0.39, 0.43 for calibration and validation datasets, respectively. The PLSR score-based multivariate models outperformed solo multivariate and optimized index-based models. Our study suggested that VNIRS with PLSR combined multivariate models can reliably be used for fast and non-invasive estimation of SOC.

Machine learning-based multi-target regression to effectively predict turning movements at signalized intersections

Effective prediction of turning movement counts at intersections through efficient and accurate methods is essential and needed for various applications. Commonly predictive methods require extensive data collection, calibration, and modeling efforts to estimate turning movements. In this study, three models were proposed to estimate turning movements at signalized intersections using approach volumes. Two sets of data from the United States and Canada were obtained to develop and test the proposed models. Machine learning-based regression models, including random forest regressor (RFR) and multioutput regressor (MOR) in addition to an artificial neural network (ANN) model, were developed and trained to analyze the relationship between approach volumes and corresponding turning movements. Multiple evaluation measurements were utilized to compare the models. All models produced satisfactory results. The RFR regression model outperformed the MOR model. However, the ANN model had the best performance when compared to the other models. The proposed models provide traffic engineers and planners with reliable and fast methods to estimate turning movements.

Quantitative Estimation of Rainfall from Remote Sensing Data Using Machine Learning Regression Models

To estimate rainfall from remote sensing data, three machine learning-based regression models, K-Nearest Neighbors Regression (K-NNR), Support Vector Regression (SVR), and Random Forest Regression (RFR), were implemented using MSG (Meteosat Second Generation) satellite data. Daytime and nighttime data from a rain gauge are used for model training and validation. To optimize the results, the outputs of the three models are combined using the weighted average. The combination of the three models (hereafter called Com-RSK) markedly improved the predictions. Indeed, the MAE, MBE, RMSE and correlation coefficient went from 23.6 mm, 10.0 mm, 40.6 mm and 89% for the SVR to 20.7 mm, 5.5 mm, 37.4 mm, and 94% when the models were combined, respectively. The Com-RSK is also compared to a few methods using the classification in the estimation, such as the ECST Enhanced Convective Stratiform Technique (ECST), the MMultic technique, and the Convective/Stratiform Rain Area Delineation Technique (CS-RADT). The Com-RSK show superior performance compared to ECST, MMultic and CS-RADT methods.The Com-RSK is also compared to the two products of satellite estimates, namely CMORPH and CHIRPS. The results indicate that Com-RSK performs better than CMORPH and CHIRPS according to MBE, RMSE and CC (coefficient correlation). A comparison with three types of satellite precipitation estimation products, such as global product, regional product, and near real-time product, is performed. Overall, the methodology developed here shows almost the same results as regional product methods and exhibits better results than near real-time and global product methods.

Graphene-based H-shaped biosensor with high sensitivity and optimization using ML-based algorithm

In this paper, a biosensing absorber based on phase transition material is presented. Different phases of the Ge2Sb2Te5 (GST) substrate have been studied for the suggested absorber with controllable characteristics. The structure has been examined to determine the infrared absorption characteristics. The detection of varying volumes of hemoglobin and urine biomolecules is studied. The graphene-GST material is utilized for spectrum tuning. The tuning for two distinct phases of GST material, amorphous GST and crystalline GST is examined. The results for aGST and cGST are reported in the form of absorption. Different amounts of hemoglobin and urine biomolecules are used to tune these two GST stages. Based on the wavelength shifts at these various concentrations, the sensitivity is computed. The highest achievable sensitivity for hemoglobin and urine biomolecules is 1500 nm/RIU and 1667 nm/RIU. The developed model is observed for various geometrical parameters and incidence angles, from which it is determined that the suggested structure is insensitive to wide angles between 0° and 60°. For urine biomolecules, the aGST design is more sensitive than the cGST design, but similar results are achieved for hemoglobin biomolecules. Experiments are conducted with Machine Learning-based regression models to minimize the simulation time and resource requirements of biosensor design. The findings of the trials indicate that a regression model can accurately estimate the absorption values for intermediate wavelengths with an R2 score of 0.9999.

Prediction of Student’s Performance With Learning Coefficients Using Regression Based Machine Learning Models

Prediction of Student’s Performance With Learning Coefficients Using Regression Based Machine Learning Models