Traditional Regression Methods Research Articles

Using Artificial Neural Networks to Improve CFS Week-3–4 Precipitation and 2-m Air Temperature Forecasts

Abstract Forecast skill from dynamical forecast models decreases quickly with projection time due to various errors. Therefore, postprocessing methods, from simple bias correction methods to more complicated multiple linear regression–based model output statistics, are used to improve raw model forecasts. Usually, these methods show clear forecast improvement over the raw model forecasts, especially for short-range weather forecasts. However, linear approaches have limitations because the relationship between predictands and predictors may be nonlinear. This is even truer for extended range forecasts, such as week-3–4 forecasts. In this study, neural network techniques are used to seek or model the relationships between a set of predictors and predictands, and eventually to improve week-3–4 precipitation and 2-m temperature forecasts made by the NOAA/NCEP Climate Forecast System. Benefitting from advances in machine learning techniques in recent years, more flexible and capable machine learning algorithms and availability of big datasets enable us not only to explore nonlinear features or relationships within a given large dataset, but also to extract more sophisticated pattern relationships and covariabilities hidden within the multidimensional predictors and predictands. Then these more sophisticated relationships and high-level statistical information are used to correct the model week-3–4 precipitation and 2-m temperature forecasts. The results show that to some extent neural network techniques can significantly improve the week-3–4 forecast accuracy and greatly increase the efficiency over the traditional multiple linear regression methods.

CENSORED DATA MODELING: A NOVEL ANTI-REGRESSION FRAMEWORK

Censored data, where the exact value of an observation is not fully observed, poses a challenge in statistical modelling. Traditional regression approaches often fail to adequately handle such data, leading to biased estimates and inaccurate predictions. In this study, we propose a novel anti-regression framework specifically designed for censored data modelling. The framework integrates advanced statistical techniques and incorporates mechanisms to mitigate the impact of censoring. By leveraging the information available from censored observations, our approach provides more reliable estimates and improved predictive performance compared to traditional regression methods. We validate the effectiveness of our framework through extensive simulations and real-world case studies. The results demonstrate the superiority of the proposed anti-regression framework in accurately modelling censored data, highlighting its potential for various applications in fields such as medical research, finance, and engineering. This study contributes to the advancement of statistical modelling techniques for censored data and provides a valuable tool for researchers and practitioners dealing with such data in their analyses.

A Dam Deformation Residual Correction Method for High Arch Dams Using Phase Space Reconstruction and an Optimized Long Short-Term Memory Network

Dam safety is an important basic part of national water network security. Building a dam deformation prediction model based on monitoring data is crucial to ensure dam safety. However, traditional statistical regression methods have shortcomings, such as a weak nonlinear fitting ability when constructing dam deformation monitoring and prediction models. The residual part of the statistical regression results usually contains parts that cannot be effectively explained by the linear regression method, that is usually highly variable and noisy. In this study, the phase space reconstruction method is used to smooth the residual term of the statistical regression model to eliminate noise interference. On this basis, an improved long short-term memory (LSTM) neural network is used to learn the nonlinearity contained in the residual term of the linear regression. Considering the impact of parameter selection on model performance, the gray wolf optimization (GWO) algorithm is used to determine the optimal parameters of the model for better performance. A high arch dam is used as a case study, with multiple measuring points used as research objects. The experimental results show that the phase space reconstruction can effectively smooth the high-frequency components in the residual term and remove noise interference. In addition, the GWO algorithm can effectively determine the hyperparameters of the LSTM network, thereby constructing a residual prediction model with high prediction accuracy. The combination of statistical models and deep learning prediction methods can effectively improve the model prediction performance while preserving the model interpretability and transparency.

Unraveling the Linkages between Molecular Abundance and Stable Carbon Isotope Ratio in Dissolved Organic Matter Using Machine Learning

Dissolved organic matter (DOM) is a complex mixture of molecules that constitutes one of the largest reservoirs of organic matter on Earth. While stable carbon isotope values (δ13C) provide valuable insights into DOM transformations from land to ocean, it remains unclear how individual molecules respond to changes in DOM properties such as δ13C. To address this, we employed Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR MS) to characterize the molecular composition of DOM in 510 samples from the China Coastal Environments, with 320 samples having δ13C measurements. Utilizing a machine learning model based on 5199 molecular formulas, we predicted δ13C values with a mean absolute error (MAE) of 0.30‰ on the training data set, surpassing traditional linear regression methods (MAE 0.85‰). Our findings suggest that degradation processes, microbial activities, and primary production regulate DOM from rivers to the ocean continuum. Additionally, the machine learning model accurately predicted δ13C values in samples without known δ13C values and in other published data sets, reflecting the δ13C trend along the land to ocean continuum. This study demonstrates the potential of machine learning to capture the complex relationships between DOM composition and bulk parameters, particularly with larger learning data sets and increasing molecular research in the future.

Survival Analysis of Colon Cancer Data using Quantile Regression

Quantile regression emerged as an alternative and robust technique to the commonly used regression models. Even in the survival analysis, quantile regression is offering more flexible modelling of survival data without any constraints attached. Unlike traditional Cox hazards models or accelerated failure models, quantile regression does not restrict the variation of the coefficients for different quantiles. In this research we modelled and compared traditional survival regression method with quantile regression applied to colon cancer data.

PT module -A Traffic Signal Classification Model Based on Convolutional Neural Networks and Random Forests

Autonomous driving and image recognition are the hot directions of Internet development nowadays. In self-driving cars it is necessary to capture traffic signs in front of the vehicle by cameras. To ensure that the information of image recognition is correct, a set of image classification models with high accuracy should be used to classify the recognized objects in order to determine the next instruction of vehicle operation. There were many achievements in the research work of traffic sign recognition, but there are still some shortcomings. By combining CNN and random forests with PT module (a module that can improve the accuracy of feature extraction), we finally came up with a classification model that can efficiently place the received traffic sign images into a specified category, which we obtained 97% accuracy on the GTSRB dataset, which is much more accurate than traditional neural network methods or regression methods. We have also evaluated it on other datasets and the results obtained are more promising.

Prediction model for gestational diabetes mellitus using the XG Boost machine learning algorithm.

To develop the extreme gradient boosting (XG Boost) machine learning (ML) model for predicting gestational diabetes mellitus (GDM) compared with a model using the traditional logistic regression (LR) method. A case-control study was carried out among pregnant women, who were assigned to either the training set (these women were recruited from August 2019 to November 2019) or the testing set (these women were recruited in August 2020). We applied the XG Boost ML model approach to identify the best set of predictors out of a set of 33 variables. The performance of the prediction model was determined by using the area under the receiver operating characteristic (ROC) curve (AUC) to assess discrimination, and the Hosmer-Lemeshow (HL) test and calibration plots to assess calibration. Decision curve analysis (DCA) was introduced to evaluate the clinical use of each of the models. A total of 735 and 190 pregnant women were included in the training and testing sets, respectively. The XG Boost ML model, which included 20 predictors, resulted in an AUC of 0.946 and yielded a predictive accuracy of 0.875, whereas the model using a traditional LR included four predictors and presented an AUC of 0.752 and yielded a predictive accuracy of 0.786. The HL test and calibration plots show that the two models have good calibration. DCA indicated that treating only those women whom the XG Boost ML model predicts are at risk of GDM confers a net benefit compared with treating all women or treating none. The established model using XG Boost ML showed better predictive ability than the traditional LR model in terms of discrimination. The calibration performance of both models was good.

Machine learning models in predicting graft survival in kidney transplantation: meta-analysis

The variations in outcome and frequent occurrence of kidney allograft failure continue to pose important clinical and research challenges despite recent advances in kidney transplantation. The aim of this systematic review was to examine the current application of machine learning models in kidney transplantation and perform a meta-analysis of these models in the prediction of graft survival. This review was registered with the PROSPERO database (CRD42021247469) and all peer-reviewed original articles that reported machine learning model-based prediction of graft survival were included. Quality assessment was performed by the criteria defined by Qiao and risk-of-bias assessment was performed using the PROBAST tool. The diagnostic performance of the meta-analysis was assessed by a meta-analysis of the area under the receiver operating characteristic curve and a hierarchical summary receiver operating characteristic plot. A total of 31 studies met the inclusion criteria for the review and 27 studies were included in the meta-analysis. Twenty-nine different machine learning models were used to predict graft survival in the included studies. Nine studies compared the predictive performance of machine learning models with traditional regression methods. Five studies had a high risk of bias and three studies had an unclear risk of bias. The area under the hierarchical summary receiver operating characteristic curve was 0.82 and the summary sensitivity and specificity of machine learning-based models were 0.81 (95 per cent c.i. 0.76 to 0.86) and 0.81 (95 per cent c.i. 0.74 to 0.86) respectively for the overall model. The diagnostic odds ratio for the overall model was 18.24 (95 per cent c.i. 11.00 to 30.16) and 29.27 (95 per cent c.i. 13.22 to 44.46) based on the sensitivity analyses. Prediction models using machine learning methods may improve the prediction of outcomes after kidney transplantation by the integration of the vast amounts of non-linear data.

Forecasting accuracy of machine learning and linear regression: evidence from the secondary CAT bond market

AbstractThe main challenge in empirical asset pricing is forecasting the future value of assets traded in financial markets with a high level of accuracy. Because machine learning methods can model relationships between explanatory and dependent variables based on complex, non-linear, and/or non-parametric structures, it is not surprising that machine learning approaches have shown promising forecasting results and significantly outperform traditional regression methods. Corresponding results were achieved for CAT bond premia forecasts in the primary market. However, since secondary market data sets have a panel data structure, it is unclear whether the results of primary market studies can be applied to the secondary market. Against this background, this study aims to build the first out-of-sample forecasting model for CAT bond premia in the secondary market, comparing different modeling approaches. We apply random forest and neural networks as representatives of machine learning methods and linear regression based on a comprehensive data set of CAT bond issues and across various forecasting settings and show that random forest forecasts are significantly more precise. Because the lack of transparency of machine learning methods may limit their applicability, especially for institutional investors, we show ways to identify important variables in the context of random forest price forecasting.

Potable Water Quality Prediction Using Artificial Intelligence and Machine Learning Algorithms for Better Sustainability

Water is one of the most important resources for human life and health. Global climate change, industrialization and urbanization pose serious dangers to existing water resources. Water quality has traditionally been predicted by expensive, time-consuming laboratory and statistical analysis. However, machine learning algorithms can be applied to determine the water quality index in real time efficiently and quickly. With this motivation, a dataset obtained from the Kaggle website was used to classify water quality in this research. Some features were found to be empty in the data set. Traditional methods (drop, mean imputation) and regression method were applied for null values. After the null values were completed, RF, Adaboost and XGBoost were applied for binary classification. Gridsearch and Randomsearch methods have been applied in hyper parameter optimization. Among all the algorithms used, the SXH hybrid method created with the Support Vector Regression (SVR) and XGBoost methods showed the best classification performance with 99.4% accuracy and F1-score. Comparison of our results with previous similar studies showed that our SVR XGboost Hybrid (SXH) model had the best performance ratio (Accuracy, F1-score). The performance of our proposed model is proof that hybrid machine learning methods can provide an innovative perspective on potable water quality.

Treatment Effect Performance of the X-Learner in the Presence of Confounding and Non-Linearity

This study critically evaluates a recent machine learning method called the X-Learner, that aims to estimate treatment effects by predicting counterfactual quantities. It uses information from the treated group to predict counterfactuals for the control group and vice versa. The problem is that studies have either only been applied to real world data without knowing the ground truth treatment effects, or have not been compared with the traditional regression methods for estimating treatment effects. This study therefore critically evaluates this method by simulating various scenarios that include observed confounding and non-linearity in the data. Although the regression X-Learner performs just as well as the traditional regression model, the other base learners performed worse. Additionally, when non-linearity was introduced into the data, the results of the X-Learner became inaccurate.

Association of physical activity with loss of knee joint space width over two years: a compositional data analysis in the Osteoarthritis Initiative

There is continued debate as to how engaging in physical activity (PA), including moderate-to-vigorous PA (MVPA), light PA (LPA), and sedentary time (SED), affects one's risk for knee osteoarthritis (OA). Traditional regression methods do not account for the codependence of these categories of PA, whereby when one category increases, the others must decrease. Thus, we used compositional data analysis (CoDA) to examine time spent in each category of PA, or PA composition, and its association with loss of knee joint space width (JSW), a common indicator of knee OA progression. We performed a secondary analysis of data from a subset of participants in the Osteoarthritis Initiative. These participants had minute-by-minute activity data collected over 7 days at baseline; we then categorized each minute as MVPA, LPA, or SED. Our exposure, PA composition, represented min/day spent in each category. Our outcome, medial JSW loss, was the difference in medial tibiofemoral JSW from baseline to 2years later. We employed CoDA, using an isometric log-ratio transformation, to examine the association of PA composition with medial JSW loss over 2years, adjusting for potential confounders. We included 969 participants (age: 64.5 years, 56% female, body mass index [BMI]: 28.8kg/m2). Mean PA composition was: MVPA 9.1min/day, LPA 278min/day, SED 690min/day. Per adjusted regression models, higher MVPA was not associated with greater medial JSW loss (β=-0.0005, P=0.97), nor was LPA (β=0.06, P=0.27) or SED (β=-0.06, P=0.21). Using CoDA, PA composition was not associated with medial JSW loss over 2years.

Tree-level almond yield estimation from high resolution aerial imagery with convolutional neural network.

Estimating and understanding the yield variability within an individual field is critical for precision agriculture resource management of high value tree crops. Recent advancements in sensor technologies and machine learning make it possible to monitor orchards at very high spatial resolution and estimate yield at individual tree level. This study evaluates the potential of utilizing deep learning methods to predict tree-level almond yield with multi-spectral imagery. We focused on an almond orchard with the 'Independence' cultivar in California, where individual tree harvesting and yield monitoring was conducted for ~2,000 trees and summer aerial imagery at 30cm was acquired for four spectral bands in 2021. We developed a Convolutional Neural Network (CNN) model with a spatial attention module to take the multi-spectral reflectance imagery directly for almond fresh weight estimation at the tree level. The deep learning model was shown to predict the tree level yield very well, with a R2 of 0.96 (±0.002) and Normalized Root Mean Square Error (NRMSE) of 6.6% (±0.2%), based on 5-fold cross validation. The CNN estimation captured well the patterns of yield variation between orchard rows, along the transects, and from tree to tree, when compared to the harvest data. The reflectance at the red edge band was found to play the most important role in the CNN yield estimation. This study demonstrates the significant improvement of deep learning over traditional linear regression and machine learning methods for accurate and robust tree level yield estimation, highlighting the potential for data-driven site-specific resource management to ensure agriculture sustainability.

Deep learning regression for quantitative LIBS analysis

One of the most promising innovation strategies for sorting and recycling post-consumer aluminium scrap is using quantitative Laser-Induced Breakdown Spectroscopy (LIBS) analysis. However, existing methods to estimate alloying element concentrations based on LIBS spectra, such as linear univariate regression and Machine Learning models, are still too limited in their performance to achieve the accuracy demanded by the industry. Therefore, this study presents novel Deep Learning approaches and compares their performance to those of traditional univariate regression and Machine Learning methods in terms of RMSE, MAE, and R2 value. For this evaluation, two sample sets of aluminium pieces are used: one containing 27 certified aluminium reference samples and the second containing 733 post-consumer scrap pieces for which the ground truth concentrations are determined by X-Ray Fluorescence (XRF).Adopting multiple loss functions, one for each element, has proven its significant value for the regression performance. It improves the results for all performance metrics in the Scrap Sample set, and the same is true for the Reference Sample set, except for the coefficient of determination of Fe, Mn and Mg. In addition, the proposed methodology considers the learning prioritisation problem to prevent that learning the concentration of the base element is prioritised over the alloying elements. Although the effect of excluding the base alloy aluminium from the learning is small and not always positive for the performance, demonstrating this effect is also considered valuable. Since the average RMSE on the prediction is just 0.02 wt% for Al and Si, and not more than 0.01 wt% for Fe, Cu, Mn, Mg, and Zn, the best-performing Deep Learning model shows promise for the future of LIBS in metal sorting applications.

Examining the use of decision trees in population health surveillance research: an application to youth mental health survey data in the COMPASS study.

In population health surveillance research, survey data are commonly analyzed using regression methods; however, these methods have limited ability to examine complex relationships. In contrast, decision tree models are ideally suited for segmenting populations and examining complex interactions among factors, and their use within health research is growing. This article provides a methodological overview of decision trees and their application to youth mental health survey data. The performance of two popular decision tree techniques, the classification and regression tree (CART) and conditional inference tree (CTREE) techniques, is compared to traditional linear and logistic regression models through an application to youth mental health outcomes in the COMPASS study. Data were collected from 74 501 students across 136 schools in Canada. Anxiety, depression and psychosocial well-being outcomes were measured along with 23 sociodemographic and health behaviour predictors. Model performance was assessed using measures of prediction accuracy, parsimony and relative variable importance. Decision tree and regression models consistently identified the same sets of most important predictors for each outcome, indicating a general level of agreement between methods. Tree models had lower prediction accuracy but were more parsimonious and placed greater relative importance on key differentiating factors. Decision trees provide a means of identifying high-risk subgroups to whom prevention and intervention efforts can be targeted, making them a useful tool to address research questions that cannot be answered by traditional regression methods.

Prediction model for missed abortion of patients treated with IVF-ET based on XGBoost: a retrospective study.

In this study, we established a model based on XGBoost to predict the risk of missed abortion in patients treated with in vitro fertilization-embryo transfer (IVF-ET), evaluated its prediction ability, and compared the model with the traditional logical regression model. We retrospectively collected the clinical data of 1,017 infertile women treated with IVF-ET. The independent risk factors were screened by performing a univariate analysis and binary logistic regression analysis, and then, all cases were randomly divided into the training set and the test set in a 7:3 ratio for constructing and validating the model. We then constructed the prediction models by the traditional logical regression method and the XGBoost method and tested the prediction performance of the two models by resampling. The results of the binary logistic regression analysis showed that several factors, including the age of men and women, abnormal ovarian structure, prolactin (PRL), anti-Müllerian hormone (AMH), activated partial thromboplastin time (APTT), anticardiolipin antibody (ACA), and thyroid peroxidase antibody (TPO-Ab), independently influenced missed abortion significantly (P<0.05). The area under the receiver operating characteristic curve (AUC) score and the F1 score with the training set of the XGBoost model (0.877 ± 0.014 and 0.730 ± 0.019, respectively) were significantly higher than those of the logistic model (0.713 ± 0.013 and 0.568 ± 0.026, respectively). In the test set, the AUC and F1 scores of the XGBoost model (0.759 ± 0.023 and 0.566 ± 0.042, respectively) were also higher than those of the logistic model (0.695 ± 0.030 and 0.550 ± 049, respectively). We established a prediction model based on the XGBoost algorithm, which can accurately predict the risk of missed abortion in patients with IVF-ET. This model performed better than the traditional logical regression model.

A personalized risk stratification tool for perinatal morbidity and mortality using explainable artificial intelligence (AI)

Late preterm delivery is frequently considered when competing maternal risks begin to outweigh perinatal risks, but tools for precise, personalized perinatal risk estimation are lacking. Traditional regression methods do not capture the complex interactions specific to each possible combination of risk factors, which can be synergistic rather than simply additive. AI-based Probabilistic Graphical Models (PGMs) can quantify these synergies transparently. We aimed to develop an ‘explainable AI’ machinery for personalized perinatal morbidity risk estimation using variables from clinical and social determinants of health domains.

Just-In-Time Learning Based Functional Spectral Data Modeling for In-Situ Measurement of Slurry Component Concentrations via Infrared Spectroscopy

For using the infrared spectroscopy to conduct in-situ measurement of slurry component concentrations during batch crystallization or fermentation process in engineering practice, a novel just-in-time learning (JITL) based functional modelling method is proposed for spectral data analysis in this paper to improve on-line measurement accuracy, by addressing engineering issues of very limited assay samples, spectral absorption nonlinearity and high-dimensional spectral variables. The traditional univariate or multivariate partial least-squares (PLS) model is subtly extended to the corresponding functional form for describing the relationship between the measured spectra and slurry component concentrations, such that the salient problem of only a small number of slurry component concentration samples available in practice for spectral data modeling could be effectively solved. The wavelet functions are adopted to properly approximate the measured spectral curves owing to their multi-scale and orthogonal properties. Accordingly, a functional spectral data model is established by wavelet functional PLS for on-line measurement of single and multiple component concentrations, respectively. Meanwhile, a JITL strategy is introduced to select the most representative samples for such modeling, which could efficiently handle the disparity issue involved with real-time samples and guarantee the model validity. Experimental results show that root mean square error of model prediction on the solution concentration during L-glutamic acid cooling crystallization process and multiple componts of ethanol fermentation process, could be evidently reduced over 40% and 50%, respectively, compared with the traditional PLS and support vector regression methods.

Feature selection and association rule learning identify risk factors of malnutrition among Ethiopian schoolchildren.

Previous studies have sought to identify risk factors for malnutrition in populations of schoolchildren, depending on traditional logistic regression methods. However, holistic machine learning (ML) approaches are emerging that may provide a more comprehensive analysis of risk factors. This study employed feature selection and association rule learning ML methods in conjunction with logistic regression on epidemiological survey data from 1,036 Ethiopian school children. Our first analysis used the entire dataset and then we reran this analysis on age, residence, and sex population subsets. Both logistic regression and ML methods identified older childhood age as a significant risk factor, while females and vaccinated individuals showed reduced odds of stunting. Our machine learning analyses provided additional insights into the data, as feature selection identified that age, school latrine cleanliness, large family size, and nail trimming habits were significant risk factors for stunting, underweight, and thinness. Association rule learning revealed an association between co-occurring hygiene and socio-economical variables with malnutrition that was otherwise missed using traditional statistical methods. Our analysis supports the benefit of integrating feature selection methods, association rules learning techniques, and logistic regression to identify comprehensive risk factors associated with malnutrition in young children.

A Comparative Assessment of Regularized Regression Techniques for Modeling the Mechanical Properties of Rubberized Concrete

Background: Over the last few decades, many researchers have investigated the properties and behavior of concrete mixtures incorporating rubber-based solid wastes as a partial substitution of natural aggregates. Within this context, they have conducted experimental studies and developed numerical models that simulate the nature of rubberized concrete. Some of these mathematical simulations were intended to provide a rapid mixture of proportioning approaches and property estimation methods. Currently, it is believed that regression analysis provides an effective tool to simply construct a mathematical expression that models a set of data. For that reason, multiple linear regression was extensively utilized in predicting rubberized concrete properties in the literature. However, the performances of regularized regression analysis approaches were not evaluated even though they provide better alternatives to traditional regression methods in terms of controlling the overfitting issue. Objective: This study aims to assess the performance of Ridge, Lasso, and elastic net regression models in estimating the compressive and tensile strengths, and modulus of elasticity of rubberized concrete. Additionally, it intends to benchmark their capabilities against the traditional multiple linear regression method. Methods: Multiple linear regression, Ridge regression, Lasso regression, ElasticNet regression, Bayesian ridge regression, Stochastic gradient descent, Huber regression, and Quantile regression methods were used in the study. Result: In general, the research findings illustrated the superior performance of regression assessment in modeling the mechanical properties of rubberized concrete. Conclusion: Indeed rubberized concrete mechanical properties can be better modeled using regularized regression techniques, such as ElasticNet-based SGD compared to traditional methods, such as MLR.