Ridge Regression Estimator Research Articles

33 Setting the Standard: Using a National Burn Registry to Benchmark Risk Adjusted Mortality

IntroductionReports of single center experience and studies of larger databases have identified several predictors of burn center mortality, including age, burn size, and inhalation injury. None of these analyses has been broad enough to allow benchmarking across burn centers. The purpose of this study was to derive a reliable risk-adjusted, statistical model of mortality based on real-life experience at many of the burn centers in the United States.MethodsWe used a national burn registry to identify 128,252 initial admissions from July 2015 through June 2020 across 103 unique burn centers. We selected 23 predictor variables, from over 50 recorded in the dataset based on completeness (at least 75% complete required) and clinical significance. Missing data were multiply imputed with a Bayesian Ridge Regression estimator. All data analysis was performed in Python using Numpy and Scikit-Learn libraries. We used Gradient boosted regression (CatBoost), a form of machine learning, to predict mortality and compared this to traditional logistic regression. Model performance was evaluated with AUC and PR curves. Using the CatBoost predictions, observe to expected mortality was calculated for each center. Confidence intervals for O/E analysis in the case of mortality prediction were calculated using a custom implementation of the Clopper-Pearson method. Analyses were run on three cohorts: All patients; Patients with 10-20% TBSA; and >20% TBSA.ResultsThe CatBoost model achieved a test AUC of 0.982 with an average precision of 0.801. The logistic regression, by comparison, produced an AUC of 0.974 with an average precision of 0.726. While accuracy, the measure most reported in the literature, is near ceiling for both models, the CatBoost model is markedly more sensitive, leading to a substantial improvement in average precision. Because of the superiority of the CatBoost model with respect to outcome prediction, we only used CatBoost models for calculation of O/E ratio (Fig. 1).ConclusionsGradient boosted regression models provided greater model performance than traditional, multivariate, logistic regression. Using data from a national burn registry, we can predict burn mortality across contributing centers allowing for meaningful O/E ratios. Further, this allows for comparison of mortality across centers contributing data to the registry.

34 Using a National Burn Registry to Develop a Model for Risk-adjusted Length of Stay Benchmarking

IntroductionLength of stay (LOS) is a frequently reported outcome after a burn injury. Previous literature estimates LOS at 1 day per % total burn surface area (TBSA) but this varies considerably across patients & centers. LOS benchmarking will benefit individual burn centers as a way to measure their performance & set expectations for patients. We sought to create a nationwide, risk-adjusted model to allow for LOS benchmarking based on data from a national burn registry.MethodsUsing data from a national burn registry, we queried admissions from 7/2015-6/2020 & identified 126,129 records with LOS data reported by 103 centers. We selected 23 predictor variables on the basis of completeness (min. 75% required) & clinical significance. Missing data were multiply imputed with a Bayesian Ridge Regression estimator. All statistics were calculated in Python using Numpy & Scikit-Learn libraries. Comparisons of unpenalized linear regression & Gradient boosted (CatBoost) regressor models were performed by measuring the R2 & concordance correlation coefficient (CCC) on the application of the model to the test dataset. The CatBoost model applied to bootstrapped versions of the entire dataset was then used to calculate O/E ratios for individual burn centers. Confidence intervals (CI) for O/E ratios were calculated using a normal distribution parametric model. Analyses were run on 3 cohorts: all patients, 10-20% TBSA, >20% TBSA.ResultsThe CatBoost model outperformed the linear regression model with a test R2 of 0.68 & CCC of 0.81 compared to the regression model with R2=0.52, CCC=0.70. The CatBoost was also less biased for higher & lower LOS durations. Due to the CatBoost model’s superiority in predicting the outcome, this model alone was used for O/E ratio calculations. The O/E ratio data from the model for all 3 cohorts are shown in Figure 1.ConclusionsGradient boosted regression models provided greater model performance than traditional regression analysis. Using national burn data, we can predict LOS across contributing burn centers while accounting for patient & center characteristics, producing more meaningful O/E ratios. These models provide a risk-adjusted LOS benchmarking using a robust data source, the first of its kind, for burn centers.

Examining factors affecting delayed completion of adjuvant chemo for patients with breast cancer: Development of ridge logistic panel estimators

The problem of multicollinearity among predictor (independent) variables is a frequent issue in logistic panel data analysis. The model parameters are estimated via the conditional maximum likelihood and unconditional maximum likelihood estimators. In this context, this paper proposes a ridge regression estimation via shrinkage methods to analyze such data. Furthermore, in view of obtaining more efficient estimators, we propose ridge estimators using different shrinkage parameters for the fixed effects logistic panel data model. An application is also presented to assess the performance of the proposed ridge estimators. The most significant factors that affect delayed completion of adjuvant chemotherapy in patients with breast cancer plus their existing outcomes in order to shed light on the link between chemotherapy duration and its outcomes according to breast cancer are illustrated in the study. The study results show that the conditional fixed effects logit estimator is more efficient and better than the unconditional pooling and unconditional fixed effects logit estimators. Moreover, we find that there are very influential factors that affected delayed completion of adjuvant chemotherapy such as Body Surface Area (BSA), Hemoglobin (HGB), Alanine Transaminase (ALT) and Creatinine (SRCR).

Modified Kibria-Lukman (MKL) estimator for the Poisson Regression Model: application and simulation.

Background: Multicollinearity greatly affects the Maximum Likelihood Estimator (MLE) efficiency in both the linear regression model and the generalized linear model. Alternative estimators to the MLE include the ridge estimator, the Liu estimator and the Kibria-Lukman (KL) estimator, though literature shows that the KL estimator is preferred. Therefore, this study sought to modify the KL estimator to mitigate the Poisson Regression Model with multicollinearity. Methods: A simulation study and a real-life study was carried out and the performance of the new estimator was compared with some of the existing estimators. Results: The simulation result showed the new estimator performed more efficiently than the MLE, Poisson Ridge Regression Estimator (PRE), Poisson Liu Estimator (PLE) and the Poisson KL (PKL) estimators. The real-life application also agreed with the simulation result. Conclusions: In general, the new estimator performed more efficiently than the MLE, PRE, PLE and the PKL when multicollinearity was present.

A High-Dimensional Counterpart for the Ridge Estimator in Multicollinear Situations

The ridge regression estimator is a commonly used procedure to deal with multicollinear data. This paper proposes an estimation procedure for high-dimensional multicollinear data that can be alternatively used. This usage gives a continuous estimate, including the ridge estimator as a particular case. We study its asymptotic performance for the growing dimension, i.e., p→∞ when n is fixed. Under some mild regularity conditions, we prove the proposed estimator’s consistency and derive its asymptotic properties. Some Monte Carlo simulation experiments are executed in their performance, and the implementation is considered to analyze a high-dimensional genetic dataset.

A restricted gamma ridge regression estimator combining the gamma ridge regression and the restricted maximum likelihood methods of estimation

In this article, we propose a restricted gamma ridge regression estimator (RGRRE) by combining the gamma ridge regression (GRR) and restricted maximum likelihood estimator (RMLE) to combat multicollinearity problem for estimating the parameter in the gamma regression model. The properties of the new estimator are discussed, and its superiority over the GRR, RMLE and traditional maximum likelihood estimator is theoretically analysed under different conditions. We also suggest some estimating methods to find the optimal value of the shrinkage parameter. A Monte Carlo simulation study is conducted to judge the performance of the proposed estimator. Finally, an empirical application is analysed to show the benefit of RGRRE over the existing estimators.

Analysis of Potential Factors Influencing China’s Regional Sustainable Economic Growth

The purpose of this article is to screen out the most important factors affecting China’s economic growth. Based on a literature review and relevant financial theoretical knowledge, China’s economic growth factors are selected from international and domestic aspects. Four methods, including least squares estimation, stepwise regression, ridge regression estimation, and Lasso regression, are used to screen and optimize 12 variables and analyze the degrees of influence empirically. The study finds that consumption levels and the development of the tertiary industry play significant roles in promoting China’s economic growth. Additionally, financial development and industrialization promote China’s economic growth, although in a gradual manner.

Modified Robust Ridge M-Estimators in Two-Parameter Ridge Regression Model

The methods of two-parameter ridge and ordinary ridge regression are very sensitive to the presence of the joint problem of multicollinearity and outliers in the y-direction. To overcome this problem, modified robust ridge M-estimators are proposed. The new estimators are then compared with the existing ones by means of extensive Monte Carlo simulations. According to mean squared error (MSE) criterion, the new estimators outperform the least square estimator, ridge regression estimator, and two-parameter ridge estimator in many considered scenarios. Two numerical examples are also presented to illustrate the simulation results.

Unbiased K-L estimator for the linear regression model

Background: In the linear regression model, the ordinary least square (OLS) estimator performance drops when multicollinearity is present. According to the Gauss-Markov theorem, the estimator remains unbiased when there is multicollinearity, but the variance of its regression estimates become inflated. Estimators such as the ridge regression estimator and the K-L estimators were adopted as substitutes to the OLS estimator to overcome the problem of multicollinearity in the linear regression model. However, the estimators are biased, though they possess a smaller mean squared error when compared to the OLS estimator. Methods: In this study, we developed a new unbiased estimator using the K-L estimator and compared its performance with some existing estimators theoretically, simulation wise and by adopting real-life data. Results: Theoretically, the estimator even though unbiased also possesses a minimum variance when compared with other estimators. Results from simulation and real-life study showed that the new estimator produced smaller mean square error (MSE) and had the smallest mean square prediction error (MSPE). This further strengthened the findings of the theoretical comparison using both the MSE and the MSPE as criterion. Conclusions: By simulation and using a real-life application that focuses on modelling, the high heating values of proximate analysis was conducted to support the theoretical findings. This new method of estimation is recommended for parameter estimation with and without multicollinearity in a linear regression model.

Unbiased K-L estimator for the linear regression model.

Background: In the linear regression model, the ordinary least square (OLS) estimator performance drops when multicollinearity is present. According to the Gauss-Markov theorem, the estimator remains unbiased when there is multicollinearity, but the variance of its regression estimates become inflated. Estimators such as the ridge regression estimator and the K-L estimators were adopted as substitutes to the OLS estimator to overcome the problem of multicollinearity in the linear regression model. However, the estimators are biased, though they possess a smaller mean squared error when compared to the OLS estimator. Methods:In this study, we developed a new unbiased estimator using the K-L estimator and compared its performance with some existing estimators theoretically, simulation wise and by adopting real-life data. Results: Theoretically, the estimator even though unbiased also possesses a minimum variance when compared with other estimators. Results from simulation and real-life study showed that the new estimator produced smaller mean square error (MSE) and had the smallest mean square prediction error (MSPE). This further strengthened the findings of the theoretical comparison using both the MSE and the MSPE as criterion. Conclusions: By simulation and using a real-life application that focuses on modelling, the high heating values of proximate analysis was conducted to support the theoretical findings. This new method of estimation is recommended for parameter estimation with and without multicollinearity in a linear regression model.

New robust ridge estimators for the linear regression model with outliers

The ridge regression estimator (RRE) is a widely used estimation method for the multiple linear regression model when the explanatory variables are correlated. The situation becomes problematic for the RRE when the data set contains outliers in the y-direction. The use of the RRE in the presence of outliers may have some adverse effects on parameter estimates. To address this issue, the robust ridge estimators based on M-estimator are available in the literature which are less sensitive to the presence of outliers. It is a well-known fact that the selection of ridge parameter k is very crucial while using the RRE and the same phenomenon may happen in the case of robust ridge estimators. This study proposes some robust ridge estimators for the ridge parameter k. The performance of proposed estimators is evaluated with the help of the Monte Carlo simulations and a real application where the mean squared error (MSE) is considered as a performance evaluation criterion. Results show a better performance of the proposed robust ridge estimators as compared to the RRE, least square and M-estimation methods. While for modified ridge M-estimator, different ridge parameters found to be better for different conditions.

Regression Analysis of Collinear Data using r-k Class Estimator: Socio-Economic and Demographic Factors Affecting the Total Fertility Rate (TFR) in India

A basic assumption concerned with general linear regression model is that there is no correlation (or no multicollinearity) between the explana tory variables. When this assumption is not satisfied, the least squares estimators have large variances and become unstable and may have a wrong sign. Therefore, we resort to biased regression methods, which stabilize the parameter estimates. Ridge regression (RR) and principal component regression (PCR) are two of the most popular biased regression methods which can be used in case of multicollinearity. But the r-k class estimator, which is composed by combining the RR estimator and the PCR estimator into a single estimator gives the better estimates of the regression coefficients than the RR estimator and PCR estimator. This paper explores the multiple regression technique using r-k class estimator between TFR and other socio-economic and demographic variables and the data has been taken from the National Family Health Survey-III (NFHS-III): 29 states of India. The analysis shows that use of contraceptive devices shares the greatest impact on fertility rate followed by maternal care, use of improved water, female age at marriage and spacing between births.

Modified Kibria-Lukman (MKL) estimator for the Poisson Regression Model: application and simulation

Background: Multicollinearity greatly affects the Maximum Likelihood Estimator (MLE) efficiency in both the linear regression model and the generalized linear model. Alternative estimators to the MLE include the ridge estimator, the Liu estimator and the Kibria-Lukman (KL) estimator, though literature shows that the KL estimator is preferred. Therefore, this study sought to modify the KL estimator to mitigate the Poisson Regression Model with multicollinearity. Methods: A simulation study and a real-life study were carried out and the performance of the new estimator was compared with some of the existing estimators. Results: The simulation result showed the new estimator performed more efficiently than the MLE, Poisson Ridge Regression Estimator (PRE), Poisson Liu Estimator (PLE) and the Poisson KL (PKL) estimators. The real-life application also agreed with the simulation result. Conclusions: In general, the new estimator performed more efficiently than the MLE, PRE, PLE and the PKL when multicollinearity was present.

Beta ridge regression estimators: simulation and application

The beta regression model is commonly used when analyzing data that come in the form of rates or percentages. However, a problem that may encounter when analyzing these kinds of data that has not been investigated for this model is the multicollinearity problem. It is well known that the maximum likelihood (ML) method is very sensitive to high inter-correlation among the explanatory variables. Therefore, this paper proposes some ridge estimators for the beta regression model to remedy the problem of instability of the traditional ML method and increase the efficiency of estimation. The performance of ridge estimators is compared to the ML estimator through the mean squared error (MSE) and the mean absolute error (MAE) criteria by conducting a Monte-Carlo simulation study and through an empirical application. According to the simulation and application results, the proposed estimators outperform the ML estimator in terms of MSE and MAE.

A Simulation Study of Some Restricted Estimators in Restricted Linear Regression Model

When the multicollinearity exists in linear regression model, the result of the Restricted Least Square estimator (RLS) is unstable. So that, more researchers proposed the restricted biased estimators to improve the efficiency of RLS estimator. In this paper, Some of biased restricted estimators have been introduced to study the performance of them. The simulation study has been given to compare these estimators. According to simulation study, we found that, the shrinkage restricted ridge regression (SRRE) estimator which proposed by Baber and Mustafa [1], has good properties comparing with other restricted estimators that given in this study. A Numerical example has been considered to illustrate the performance of these estimators

On the generalized biased estimators for the gamma regression model: methods and applications

The gamma regression model (GRM) is commonly used if the response variable is continuous and positively skewed. In the existence of multicollinearity problem, maximum likelihood estimator (MLE) is inadequate for estimating the GRM coefficients. To avoid this issue, well-known estimators such as, ridge and Liu are generally used. In this study, we propose the generalized class of biased estimators, namely generalized ridge, and generalized Liu estimators for the GRM with correlated explanatory variables. The standard properties of the proposed estimators are derived and illustrated using Monte Carlo simulation study and two real applications where mean squared error is considered as an assessment criterion. Based on the findings of simulation and empirical applications, we found that the performance of the generalized gamma ridge regression estimator is better as compared to MLE, and generalized gamma Liu estimator.

Modified Kibria-Lukman (MKL) estimator for the Poisson Regression Model: application and simulation.

Background: Multicollinearity greatly affects the Maximum Likelihood Estimator (MLE) efficiency in both the linear regression model and the generalized linear model. Alternative estimators to the MLE include the ridge estimator, the Liu estimator and the Kibria-Lukman (KL) estimator, though literature shows that the KL estimator is preferred. Therefore, this study sought to modify the KL estimator to mitigate the Poisson Regression Model with multicollinearity. Methods: A simulation study and a real-life study were carried out and the performance of the new estimator was compared with some of the existing estimators. Results: The simulation result showed the new estimator performed more efficiently than the MLE, Poisson Ridge Regression Estimator (PRE), Poisson Liu Estimator (PLE) and the Poisson KL (PKL) estimators. The real-life application also agreed with the simulation result. Conclusions: In general, the new estimator performed more efficiently than the MLE, PRE, PLE and the PKL when multicollinearity was present.

On the ridge estimation of the Conway‐Maxwell Poisson regression model with multicollinearity: Methods and applications

AbstractIn data analysis, count data modeling contributing a significant role. The Conway‐Maxwell Poisson (COMP) is one of the flexible count data models to deal over and under dispersion. In the COMP regression model, when the explanatory variables are correlated, then the maximum likelihood estimator does not give efficient results due to the large standard error (SE) of the estimates. To overcome the effect of multicollinearity, we have proposed some ridge regression estimators in the COMP regression model by introducing dispersion parameter in the context of overdispersion, equidispersion, and underdispersion. The Iterative reweighted least method is used for the estimation of ridge regression coefficients in the COMP regression model. To evaluate the performance of the proposed estimators, we use mean squared error (MSE) as the performance evaluation criteria. Theoretical comparison of the proposed estimators with the competitor estimators is made and conditions of efficiency have been derived. The proposed estimator is evaluated with the help of a simulation study and two real applications. The results of the simulation study and real applications show the superiority of the proposed estimator because the proposed estimator produces smaller MSE and SEs of the COMP regression estimates with multicollinearity.

Almost unbiased ridge estimator in the count data regression models

The ridge estimator has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The Poisson regression negative binomial regression models are well-known model in application when the response variable is count data. However, it is known that multicollinearity negatively affects the variance of maximum likelihood estimator of the count regression coefficients. To address this problem, a count data ridge estimator has been proposed by numerous researchers. In this paper, an almost unbiased regression estimator is proposed and derived. Our Monte Carlo simulation results suggest that the proposed estimator can bring significant improvement relative to other existing estimators. In addition, the real application results demonstrate that the proposed estimator outperforms both negative binomial ridge regression and maximum likelihood estimators in terms of predictive performance.

New restricted biased estimator based on modified unbiased ridge regression estimator

More researchers used biased estimators instead of the restricted least square estimator (RLS) that is unbiased to improve the efficiency of the estimator. Therefore, in this paper, we proposed new biased estimator, called restricted modified unbiased ridge regression (RMUR) based on modified unbiased ridge regression (MURR) that suggested by Batah and Gory (2009). By some theorems and a simulation study, we show that, this new estimator has desirable properties as the MUR and it is better than the RLS. A numerical example from literature is used to illustrate the results.