Traditional Regression Methods Research Articles

Predictive performance of presence‐only species distribution models: a benchmark study with reproducible code

AbstractSpecies distribution modeling (SDM) is widely used in ecology and conservation. Currently, the most available data for SDM are species presence‐only records (available through digital databases). There have been many studies comparing the performance of alternative algorithms for modeling presence‐only data. Among these, a 2006 paper from Elith and colleagues has been particularly influential in the field, partly because they used several novel methods (at the time) on a global data set that included independent presence–absence records for model evaluation. Since its publication, some of the algorithms have been further developed and new ones have emerged. In this paper, we explore patterns in predictive performance across methods, by reanalyzing the same data set (225 species from six different regions) using updated modeling knowledge and practices. We apply well‐established methods such as generalized additive models and MaxEnt, alongside others that have received attention more recently, including regularized regressions, point‐process weighted regressions, random forests, XGBoost, support vector machines, and the ensemble modeling framework biomod. All the methods we use include background samples (a sample of environments in the landscape) for model fitting. We explore impacts of using weights on the presence and background points in model fitting. We introduce new ways of evaluating models fitted to these data, using the area under the precision‐recall gain curve, and focusing on the rank of results. We find that the way models are fitted matters. The top method was an ensemble of tuned individual models. In contrast, ensembles built using the biomod framework with default parameters performed no better than single moderate performing models. Similarly, the second top performing method was a random forest parameterized to deal with many background samples (contrasted to relatively few presence records), which substantially outperformed other random forest implementations. We find that, in general, nonparametric techniques with the capability of controlling for model complexity outperformed traditional regression methods, with MaxEnt and boosted regression trees still among the top performing models. All the data and code with working examples are provided to make this study fully reproducible.

Bayesian network-based missing mechanism identification (BN-MMI) method in medical research

BackgroundTraditional approaches to identify missing mechanisms are usually based on the hypothesis test and confronted with both theoretical and practical challenges. It has been proved that the Bayesian network is powerful in integrating, analyzing and visualizing information, and some previous researches have verified the promising features of Bayesian network to deal with the aforementioned challenges in missing mechanism identification. Based on the above reasons, this paper explores the application of Bayesian network to the identification of missing mechanisms for the first time, and proposes a new method, the Bayesian network-based missing mechanism identification (BN-MMI) method, to identify missing mechanism in medical research.MethodsThe procedure of BN-MMI method consists three easy-to-implement steps: estimating the missing data structure by the Bayesian network; assessing the credibility of the estimated missing data structure; and identifying the missing mechanism from the estimated missing data structure. The BN-MMI method is verified by simulation research and empirical research.ResultsThe simulation study verified the validity, consistency and robustness of BN-MMI method, and indicated its outperformance in contrast to the traditional logistic regression method. In addition, the empirical study illustrated the applicability of BN-MMI method in the real world by an example of medical record data.ConclusionsIt was confirmed that the BN-MMI method itself, together with human knowledge and expertise, could identify the missing mechanisms according to the probabilistic dependence/independence relations among variables of interest. At the same time, our research shed light upon the potential application of BN-MMI method to a broader range of missing data issues in medical studies.

Ridge-GME estimation in linear mixed models

In this paper, we concentrate on the generalized maximum entropy (GME) estimators. The aim is to improve the problem of multicollinearity in the linear mixed models (LMMs). Then the asymptotic properties of these estimators will be derived. Also, we obtain the Ridge-GME estimators, which combines ridge regression and GME, to enhance the problem of the traditional ridge regression and GME method in these models. Finally, a simulation study and a numerical example have been conducted to show the superiority of the Ridge-GME estimator over the ridge estimator (RE) and the maximum likelihood (ML) estimators.

A causal data fusion method for the general exposure and outcome.

With the advent of the big data era, the need to combine multiple individual data sets to draw causal effects arises naturally in many medical and biological applications. Especially each data set cannot measure enough confounders to infer the causal effect of an exposure on an outcome. In this article, we extend the method proposed by a previous study to causal data fusion of more than two data sets without external validation and to a more general (continuous or discrete) exposure and outcome. Theoretically, we obtain the condition for identifiability of exposure effects using multiple individual data sources for the continuous or discrete exposure and outcome. The simulation results show that our proposed causal data fusion method has unbiased causal effect estimate and higher precision than traditional regression, meta-analysis and statistical matching methods. We further apply our method to study the causal effect of BMI on glucose level in individuals with diabetes by combining two data sets. Our method is essential for causal data fusion and provides important insights into the ongoing discourse on the empirical analysis of merging multiple individual data sources.

Predicting home sale prices: A review of existing methods and illustration of data stream methods for improved performance

AbstractThe need for accurate and unbiased assessment of residential real property has always been important not only to financial institutions lending on or holding such assets but also to municipalities that rely on property taxes as their critical source of revenue. The common methodology for predicting residential property sale price is based on traditional multiple regression in spite of known issues. Machine learning methods have been proposed as an alternative approach but the results are far from satisfactory. A review of existing studies and relevant issues can help researchers better assess the pros and cons of the approaches in this important stream of research and move the field forward. This article provides such a review. In our review, we have noticed that common to both the regression‐based methods and machine learning methods are the use of batch‐mode learning. Thus in addition to providing a review of recent research on batch‐based residential property prediction models, this article also explores a new approach to constructing residential property price prediction models by treating past sale records as an evolving data stream. The results of our study show that the data stream approach outperforms the traditional regression method and demonstrate the potential of data stream methods in improving prediction models for residential property prices.This article is categorized under: Application Areas > Business and Industry Technologies > Machine Learning Technologies > Prediction

Healthcare under pressure: modelling COVID-19 fatalities with multiscale geographically weighted regressions

PurposeThe paper aims to disentangle the factors behind territorial disparities in the coronavirus disease 2019 (COVID-19) case fatality ratio, focusing on the pressure put by the pandemic on healthcare services and adopting a spatial perspective.Design/methodology/approachMultiscale geographically weighted regression (MGWR) models have been used for uncovering the spatial variability in the impact of healthcare services on COVID-19 case fatality ratio, allowing authors to better capture the real spatial patterns at local level. The authors proved that this approach yields better results, and the MGWR model outperforms traditional regression methods. The selected case studies are two of the biggest UE countries, among the first affected by a high incidence of COVID-19 cases, namely Italy and Germany.FindingsThe authors found sizeable regional differences in COVID-19 mortality rates within each of the analysed countries, and the stress borne by local healthcare systems seems to be the most powerful factor in explaining them. In line with other studies, the authors found additional factors of influence, such as age distribution, gender ratio, population density and regional development.Originality/valueThis research clearly indicated that COVID-19 related deaths are strongly associated with the degree of resilience of the local healthcare systems. The authors supply localized results on the factors of influence, useful for assisting the decision-makers in prioritizing limited healthcare resources. The authors provide a scientific argument in favour of the decentralization of the pandemic management towards local authorities not neglecting, however, the necessary regional or national coordination.

Predicting Within-City Variations in Ultrafine Particle and Black Carbon Concentrations in Bucaramanga, Columbia Using Open Source Data and Images

BACKGROUND AND AIM: Outdoor ultrafine particles (UFP; 100 nm) and black carbon (BC) vary greatly within cities and may have adverse impacts on human health such as cardiovascular mortality and brain tumour incidence. Traditional regression methods for estimating spatial distributions of ambient air pollution depend on extensive, curated geospatial information system databases. This has resulted in a disparity wherein little is known about local air pollution in the data-sparse settings of low- and middle- income countries. We used a hybrid approach to develop new models to estimate within-city spatial variations in outdoor UFP and BC concentrations across Bucaramanga, Colombia. METHODS: We conducted a large-scale mobile monitoring campaign over twenty days in 2019. Land use regression (LUR) models were developed using land use parameters from curated and open-source databases. Convolutional Neural Network (CNN) models were developed using Google Maps satellite view and street view images. Combined models were developed by combining predictions from the LUR and CNN models. After training in the training and validation sets, predictions were generated in the test set and compared to measured values. RESULTS:The combined UFP model (R2=0.54) outperformed the CNN (R2=0.47) and land use regression (LUR) models (R2=0.47) on their own. Similarly, the combined BC model also outperformed the CNN and LUR BC models (R2 = 0.51 vs 0.43 and 0.45 respectively). Spatial variations in model performance were more stable for the CNN and combined models compared to the LUR models suggesting that the combined approach may be less likely to contribute to differential exposure measurement error in epidemiological studies. CONCLUSIONS:Estimates from these models can then be applied to population-based cohorts in order to evaluate population health risks. Additionally, our findings demonstrated that satellite and street-level images can be combined with a traditional LUR modelling approach to improve predictions of within-city spatial variations in outdoor UFP and BC concentrations. KEYWORDS: Ultrafine particles, Black Carbon, Deep learning, Images, Land use regression

Enhanced prediction of atrial fibrillation and mortality among patients with congenital heart disease using nationwide register-based medical hospital data and neural networks.

To improve short-and long-term predictions of mortality and atrial fibrillation (AF) among patients with congenital heart disease (CHD) from a nationwide population using neural networks (NN). The Swedish National Patient Register and the Cause of Death Register were used to identify all patients with CHD born from 1970 to 2017. A total of 71 941 CHD patients were identified and followed-up from birth until the event or end of study in 2017. Based on data from a nationwide population, a NN model was obtained to predict mortality and AF. Logistic regression (LR) based on the same data was used as a baseline comparison. Of 71 941 CHD patients, a total of 5768 died (8.02%) and 995 (1.38%) developed AF over time with a mean follow-up time of 16.47 years (standard deviation 12.73 years). The performance of NN models in predicting the mortality and AF was higher than the performance of LR regardless of the complexity of the disease, with an average area under the receiver operating characteristic of >0.80 and >0.70, respectively. The largest differences were observed in mortality and complexity of CHD over time. We found that NN can be used to predict mortality and AF on a nationwide scale using data that are easily obtainable by clinicians. In addition, NN showed a high performance overall and, in most cases, with better performance for prediction as compared with more traditional regression methods.

Support vector regression for high-resolution beach surface moisture estimation from terrestrial LiDAR intensity data

Beach Surface Moisture (BSM) is a key attribute in the coastal investigations of land-atmospheric water and energy fluxes, groundwater resource budgets and coastal beach/dune development. In this study, an attempt has been made for the first time to estimate BSM from terrestrial LiDAR intensity data based on the Support Vector Regression (SVR). A long-range static terrestrial LiDAR (Riegl VZ-2000) was adopted to collect point cloud data of high spatiotemporal resolution on the Ostend-Mariakerke beach, Belgium. Based on the field moisture samples, SVR models were developed to retrieve BSM, using the backscattered intensity, scanning ranges and incidence angles as input features. The impacts of the training samples’ size and density on the predictive accuracy and generalization capability of the SVR models were fully investigated based on simulated BSM-intensity samples. Additionally, we compared the performance of the SVR models for BSM estimation with the traditional Stepwise Regression (SR) method and the Artificial Neural Network (ANN). Results show that SVR could accurately retrieve the BSM from the backscattered intensity with high reproducibility (average test RMSE of 0.71% ± 0.02% and R2 of 0.98% ± 0.002%). The Radial Basis Function (RBF) was the most suitable kernel for SVR model development in this study. The impacts of scanning geometry on the intensity could also be accurately corrected in the process of estimating BSM by the SVR models. However, compared to the SR method, the predictive accuracy and generalization performance of SVR models were significantly dependent on the training samples’ coverage, size and distribution, suggesting the need for the training samples of uniform distribution and representativeness. The minimum size of training samples required for SVR model development was 54. Under this condition, SVR performed similarly to ANN with a test RMSE of 1.06%, but SVR still performed acceptably (with an RMSE of 1.83%) even using extremely few training samples (only 16 field samples of uniform distribution), far better than the ANN (with an RMSE of 4.02%).

Hotspot Temperature Prediction of Dry-Type Transformers Based on Particle Filter Optimization with Support Vector Regression

Both poor cooling methods and complex heat dissipation lead to prominent asymmetry in transformer temperature distribution. Both the operating life and load capacity of a power transformer are closely related to the winding hotspot temperature. Realizing accurate prediction of the hotspot temperature of transformer windings is the key to effectively preventing thermal faults in transformers, thus ensuring the reliable operation of transformers and accurately predicting transformer operating lifetimes. In this paper, a hot spot temperature prediction method is proposed based on the transformer operating parameters through the particle filter optimization support vector regression model. Based on the monitored transformer temperature, load rate, transformer cooling type, and ambient temperature, the hotspot temperature of a dry-type transformer can be predicted by a support vector regression method. The hyperparameters of the support vector regression are dynamically optimized here according to the particle filter to improve the optimization accuracy. The validity and accuracy of the proposed method are verified by comparing the proposed method with a traditional support vector regression method based on the real operating data of a 35 kV dry-type transformer.

A DEEP LEARNING APPROACH FOR FORECASTING GLOBAL COMMODITIES PRICES

Forecasting future values of time-series data is a critical task in many disciplines including financial planning and decision-making. Researchers and practitioners in statistics apply traditional statistical methods (such as ARMA, ARIMA, ES, and GARCH) for a long time with varying. accuracies. Deep learning provides more sophisticated and non-linear approximation that supersede traditional statistical methods in most cases. Deep learning methods require minimal features engineering compared to other methods; it adopts an end-to-end learning methodology. In addition, it can handle a huge amount of data and variables. Financial time series forecasting poses a challenge due to its high volatility and non-stationarity nature. This work presents a hybrid deep learning model based on recurrent neural network and Autoencoders techniques to forecast commodity materials' global prices. Results showbetter accuracy compared to traditional regression methods for short-term forecast horizons (1,2,3 and 7days).

Crowd estimation using key‐point matching with support vector regression

The crowd behaviour understanding and density estimation are some of the fast-growing fields in video surveillance. There are many techniques (detection and regression) that are used as the method of crowd analysis and estimation. In the present approach, SVR (support vector regression) is used as the basic analysis technique and the novel key-point matching with SURF (speedup robust feature) is used as the feature extractor for moving objects in the video. The traditional linear regression methods used mainly key-point as one of the statistical features instead of matching with consecutive frames, but we used the magnitude of the optical flow for foreground object extraction instead of inter-frame difference. The combination of the optical flow of foreground objects and key-point matching generates new features apart from conventional features such as areas and corners. In this new approach, key-point pairing with linear regression is tested with the PETS2009 dataset, and performance is compared with the existing approaches.

Statistical upscaling of ecosystem CO2 fluxes across the terrestrial tundra and boreal domain: Regional patterns and uncertainties.

The regional variability in tundra and boreal carbon dioxide (CO2 ) fluxes can be high, complicating efforts to quantify sink-source patterns across the entire region. Statistical models are increasingly used to predict (i.e., upscale) CO2 fluxes across large spatial domains, but the reliability of different modeling techniques, each with different specifications and assumptions, has not been assessed in detail. Here, we compile eddy covariance and chamber measurements of annual and growing season CO2 fluxes of gross primary productivity (GPP), ecosystem respiration (ER), and net ecosystem exchange (NEE) during 1990-2015 from 148 terrestrial high-latitude (i.e., tundra and boreal) sites to analyze the spatial patterns and drivers of CO2 fluxes and test the accuracy and uncertainty of different statistical models. CO2 fluxes were upscaled at relatively high spatial resolution (1km2 ) across the high-latitude region using five commonly used statistical models and their ensemble, that is, the median of all five models, using climatic, vegetation, and soil predictors. We found the performance of machine learning and ensemble predictions to outperform traditional regression methods. We also found the predictive performance of NEE-focused models to be low, relative to models predicting GPP and ER. Our data compilation and ensemble predictions showed that CO2 sink strength was larger in the boreal biome (observed and predicted average annual NEE -46 and -29g Cm-2 yr-1 , respectively) compared to tundra (average annual NEE +10 and -2g Cm-2 yr-1 ). This pattern was associated with large spatial variability, reflecting local heterogeneity in soil organic carbon stocks, climate, and vegetation productivity. The terrestrial ecosystem CO2 budget, estimated using the annual NEE ensemble prediction, suggests the high-latitude region was on average an annual CO2 sink during 1990-2015, although uncertainty remains high.

Predicting the 10-year risk of cataract surgery using machine learning techniques on questionnaire data: findings from the 45 and Up Study

Background/aimsTo investigate the feasibility and accuracy of using machine learning (ML) techniques on self-reported questionnaire data to predict the 10-year risk of cataract surgery, and to identify meaningful predictors of...

OP0288 MACHINE LEARNING ALGORITHMS TO PREDICT COVID-19 ACUTE RESPIRATORY DISTRESS SYNDROME IN PATIENTS WITH RHEUMATIC DISEASES: RESULTS FROM THE GLOBAL RHEUMATOLOGY ALLIANCE PROVIDER REGISTRY

Background:Acute Respiratory Distress Syndrome (ARDS) is a life-threatening complication of COVID-19 and has been reported in approximately one-third of hospitalized patients with COVID-191. Risk factors associated with the development of ARDS include older age and diabetes2. However, little is known about factors associated with ARDS in the setting of COVID-19, in patients with rheumatic disease or those receiving immunosuppressive medications. Prediction algorithms using traditional regression methods perform poorly with rare outcomes, often yielding high specificity but very low sensitivity. Machine learning algorithms optimized for rare events are an alternative approach with potentially improved sensitivity for rare events, such as ARDS in COVID-19 among patients with rheumatic disease.Objectives:We aimed to develop a prediction model for ARDS in people with COVID-19 and pre-existing rheumatic disease using a series of machine learning algorithms and to identify risk factors associated with ARDS in this population.Methods:We used data from the COVID-19 Global Rheumatology Alliance (GRA) Registry from March 24 to Nov 1, 2020. ARDS diagnosis was indicated by the reporting clinician. Five machine learning algorithms optimized for rare events predicted ARDS using 42 variables covering patient demographics, rheumatic disease diagnoses, medications used at the time of COVID-19 diagnosis, and comorbidities. Model performance was assessed using accuracy, area under curve, sensitivity, specificity, positive predictive value, and negative predictive value. Adjusted odds ratios corresponding to the 10 most influential predictors from the best performing model were derived using hierarchical multivariate mixed-effects logistic regression that accounted for within-country correlations.Results:A total of 5,931 COVID-19 cases from 67 countries were included in the analysis. Mean (SD) age was 54.9 (16.0) years, 4,152 (70.0%) were female, and 2,399 (40.5%) were hospitalized. ARDS was reported in 388 (6.5% of total and 15.6% of hospitalized) cases. Statistically significant differences in the risk of ARDS were observed by demographics, diagnoses, medications, and comorbidities using unadjusted univariate comparisons (data not shown). Gradient boosting machine (GBM) had the highest sensitivity (0.81) and was considered the best performing model (Table 1). Hypertension, interstitial lung disease, kidney disease, diabetes, older age, glucocorticoids, and anti-CD20 monoclonal antibodies were associated with the development of ARDS while tumor necrosis factor inhibitors were associated with a protective effect (Figure 1).Table 1.Performance of machine learning algorithms.GBMSVMGLMNETNNETRFAccuracy0.790.680.660.660.67AUC0.750.700.740.580.74Sensitivity0.810.680.650.680.67Specificity0.490.600.730.480.68PPV0.960.960.970.950.97NPV0.160.120.130.090.13GBM: Gradient Boosting Machine, SVM: Support vector machines, GLMNET: Lasso and Elastic-Net Regularized Generalized Linear Models, NNET: Neural Networks, RF: Random Forest. AUC: Area Under Curve; PPV: Positive Predictive Value; NPV: Negative Predictive Value.Conclusion:In this global cohort of patients with rheumatic disease, a machine learning model, GBM, predicted the onset of ARDS with 81% sensitivity using baseline information obtained at the time of COVID-19 diagnosis. These results identify patients who may be at higher risk of severe COVID-19 outcomes. Further studies are necessary to validate the proposed prediction model in external cohorts and to evaluate its clinical utility. Disclaimer: The views expressed here are those of the authors and participating members of the COVID-19 Global Rheumatology Alliance, and do not necessarily represent the views of the ACR, NIH, (UK) NHS, NIHR, or the department of Health.

Households’ Electricity Consumption in Hungarian Urban Areas

The aim of this study is to examine the factors influencing the electricity consumption of urban households and to prove these with statistically significant results. The study includes 46 small and medium-sized towns in Hungary. The methodology of the study is mainly provided by a model that can be used for this purpose; however, the results obtained with the traditional regression method are compared with the results of another, more complex estimation method, the artificial neural network, which has the advantage of being able to use different types of models. The focus of our article is on methodological alignment, not necessarily the discovery of new results. Certain demographic characteristics significantly determine the energy demand of a household sector in a municipality. In this case, as the ratio of people aged 60 or over within a city rises by 1%, the urban household average energy consumption decreases by 61 kilowatt hours, and when it rises by 1%, the amount of pollutants expelled from urban households’ average energy consumption may decrease by 22.8745 kg. The research area of our paper was greatly influenced by the availability of the statistical data. The results can be used in the planning of urban developments.

Data-driven machine learning approach for exploring and assessing mechanical properties of carbon nanotube-reinforced cement composites

Traditional experimental investigation on the mechanical properties of cement composites is deprecated due to the intensive time and labor involved. Existing predictive models can hardly map the complicated relationships among mechanical attributes and behavior. This study first adopts machine learning to predict the mechanical properties of carbon nanotube (CNT)-reinforced cement composites. For this purpose, predictive models are trained on the previously published experimental data and results demonstrate that machine learning models present better generalization ability and predictive performance than the traditional response surface methodology. A sensitivity analysis indicates that the factor having the maximum influence on compressive strength is the length of CNTs, whereas that having the maximum influence on flexural strength is the curing temperature. Thus, it can be concluded that compared with the traditional experimental investigation and regression methods, machine learning can efficiently and accurately predict the mechanical properties of CNT-reinforced cement composites.

Landsat 8 monitoring of multi-depth suspended sediment concentrations in Lake Erie’s Maumee River using machine learning

ABSTRACT Satellite remote sensing has been widely used to map suspended sediment concentration (SSC) in waterbodies. However, due to the complexity of sediment-water interactions, it has been difficult to derive linear and non-linear regression equations to reliably predict SSC, especially when trying to estimate depth of integrated sediment. This study uses Landsat 8 OLI (Operational Land Imager) sensor to map SSC within the Maumee River in Ohio, USA, at multiple depth intervals (15, 61, 91, and 182 cm). Simple linear least squares regression (LLSR), and three common machine learning models: random forest (RF), support vector regression (SVR), and model averaged neural network (MANN) were used to estimate SSC at the depth intervals. All machine learning models significantly outperformed LLSR while RF performed the best. In both RF and MANN, R 2 (coefficient of determination) increases with depth with a maximum R 2 of 0.89 and 0.83, respectively, at a depth of 0–182 cm. The results show that machine learning models can implement nonlinear relationships that produce better predictions than traditional linear regression methods in estimating depth integrated SSC, especially when samples are limited.

Characterization of Otto Chips by Particle Swarm Optimization

Abstract Recently a surface plasmon resonance (SPR) optical sensor, based on the Otto configuration — the Otto chip — has been developed. One essential step in the quality control of the fabrication process is characterization of the active region of several devices in a batch. Characterization is done by measuring the angular spectrum of the optical re ectance on several points across the active region of the device, and determining parameters by regression analysis of the data. Traditional gradient methods used in the regression process are extremely dependent on an initial guess and are not very efficient for batch analysis of curves, when those include poorly defined SPR spectra, where an initial guess may be hard to infer. An alternative approach for the regression problem is to model the analysis as an optimization problem and using an efficient stochastic algorithm. In this paper one discusses the use of Particle Swarm Optimization (PSO) for characterization of Otto chip devices. From comparative studies carried out in an existing Otto chip, it is observed that PSO can be a very efficient approach for batch analysis and yields better results when compared with the traditional gradient-based regression method.

Robust regression with compositional covariates including cellwise outliers

We propose a robust procedure to estimate a linear regression model with compositional and real-valued explanatory variables. The proposed procedure is designed to be robust against individual outlying cells in the data matrix (cellwise outliers), as well as entire outlying observations (rowwise outliers). Cellwise outliers are first filtered and then imputed by robust estimates. Afterwards, rowwise robust compositional regression is performed to obtain model coefficient estimates. Simulations show that the procedure generally outperforms a traditional rowwise-only robust regression method (MM-estimator). Moreover, our procedure yields better or comparable results to recently proposed cellwise robust regression methods (shooting S-estimator, 3-step regression) while it is preferable for interpretation through the use of appropriate coordinate systems for compositional data. An application to bio-environmental data reveals that the proposed procedure—compared to other regression methods—leads to conclusions that are best aligned with established scientific knowledge.