Robust Ridge Regression Research Articles

Weighted penalized m-estimators in robust ridge regression: an application to gasoline consumption data

ABSTRACT The OLS and ridge regression (RR) estimators are adversely affected, when the problem of multicollinearity and y-direction outliers occur together. The robust ridge regression with penalized parameters offers biased estimators with lower variance than OLS and RR. The optimal penalized parameter value is crucial for balancing variance and bias. This study proposes three new weighted robust penalized M-estimators to address both multicollinearity and y-direction outliers. Their performance is evaluated against OLS, RR and existing penalized M-estimators using Monte Carlo simulations based on mean squared error (MSE). The new estimators exhibit lower MSE in the presence of multicollinearity and y-direction outliers. A real data application on gasoline consumption demonstrates the superior performance of the new estimators.

Estimasi Parameter Regresi Ridge Robust pada Data Profil Kesehatan Sulawesi Selatan

ABSTRACT Multicollinearity is one of the assumption violations in regression analysis. The existence of multicollinearity causes the standard error to increase. Ridge regression is one of the regression analysis approaches that can overcome this problem. Besides multicollinearity, another problem that often occurs is outliers. The existence of outliers causes the data not to be normally distributed. Ridge Robust Least Trimmed Square Regression is a method that can be used to overcome multicollinearity and outlier problems in the data simultaneously in the regression analysis model. The purpose of this study was to obtain the estimation results of the least trimmed square ridge robust regression model on the Health Profile data of South Sulawesi in 2017. From the results and discussion it was found that the least trimmed square ridge robust regression method has an Rsquare value or ?2 which is 88% and an MSE value 1.96, thus indicating that the ridge robust least trimmed square model fits better in dealing with data containing multicollinearity and outliers. Keywords: Robust Ridge Regression, Least Trimmed Square, Multicollinearity, Outlier, Infant Mortality Rate.

New penalized M-estimators in robust ridge regression: real life applications using sports and tobacco data

The ordinary least square and ridge regression estimators are very sensitive to the joint presence of multicollinearity and outliers in the y-direction. The method of robust ridge regression with penalized or biasing parameter provides biased but efficient results than least square and ridge regression estimators. The optimal choice of k m plays a key role in such a bias-variance tradeoff. In this paper, we have considered some existing estimators and also proposed some new penalized m-estimators. The new estimators have been compared with existing estimators through extensive Monte Carlo simulations. Based on the mean squared error criterion, the new penalized m-estimators outperform all the competing estimators under certain conditions. The Tobacco and Sports data sets have been used as numerical examples in the study. The findings show that the new penalized m-estimators outperform in the existing m-estimators.

The Comparison of Classical and Robust Biased Regression Methods for Determining Unemployment Rate in Turkey: Period of 1985-2012

Unemployment is one of the most important issues in macro economics. Unemployment creates many economic and social problems in the economy. The condition and qualification of labor force in a country show economical developments. In the light of these facts, a developing country should overcome the problem of unemployment. In this study, the performance of robust biased Robust Ridge Regression (RRR), Robust Principal Component Regression (RPCR) and RSIMPLS methods are compared with each other and their classical versions known as Ridge Regression (RR), Principal Component Regression (PCR) and Partial Least Squares Regression (PLSR) in terms of predictive ability by using trimmed Root Mean Squared Error (TRMSE) statistic in case of both of multicollinearity and outliers existence in an unemployment data set of Turkey. Analysis results show that RRR model is chosen as the best model for determining unemployment rate in Turkey for the period of 1985-2012. Robust biased RRR method showed that the most important independent variable effecting the unemployment rate is Purchasing Power Parities (PPP). The least important variables effecting the unemployment rate are Import Growth Rate (IMP) and Export Growth Rate (EXP). Hence, any increment in PPP cause an important increment in unemployment rate, however, any increment in IMP causes an unimportant increase in unemployment rate. Any increment in EXP causes an unimportant decrease in unemployment rate.

Estimate The Parameters in Presence of Multicollinearity And Outliers Using Bisquare Weighted Ridge Least Median Squares Regression (wrlms)

The presence of multicollinearity and outliers are classical problems of data within the linear regression framework. We are going to present a proposal of a new method which can be a potential candidate for robust ridge regression as well as a robust detection of multicollinearity. This proposal arises as a logical combination of principles used in the ridge regression and the Bisquare weighted function. The technique of the Least Median of Squares (LMS) is used for the sake of overcoming the resulting regression problems. This paper investigates the non-resistance of Ordinary Least Square (OLS) to multicollinearity and outliers and proposes the utilization of robust regression for instance, Least Median Squares LMS to detect non-normality of residuals, the use of robust methods yields more reliable trend estimations and outlier detection. LMS is introduced as a robust regression technique and through medical application its effect on regression is discussed. The numerical example and simulation study shows that the outcome of the Weighted Ridge Least Median Squares (WRLMS) is better than other estimators in terms of its efficiency. This has been done by utilizing both Standard Error (SE) and the Root Mean Squared Error criterion for the numerical example and simulation study, respectively as far as a lot of combinations of error distribution and degree of multicollinearity are concerned.

Traffic Flow Anomaly Detection Based on Robust Ridge Regression with Particle Swarm Optimization Algorithm

Traffic flow anomaly detection is helpful to improve the efficiency and reliability of detecting fault behavior and the overall effectiveness of the traffic operation. The data detected by the traffic flow sensor contains a lot of noise due to equipment failure, environmental interference, and other factors. In the case of large traffic flow data noises, a traffic flow anomaly detection method based on robust ridge regression with particle swarm optimization (PSO) algorithm is proposed. Feature sets containing historical characteristics with a strong linear correlation and statistical characteristics using the optimal sliding window are constructed. Then by providing the feature sets inputs to the PSO-Huber-Ridge model and the model outputs the traffic flow. The Huber loss function is recommended to reduce noise interference in the traffic flow. The L2 regular term of the ridge regression is employed to reduce the degree of overfitting of the model training. A fitness function is constructed, which can balance the relative size between the k-fold cross-validation root mean square error and the k-fold cross-validation average absolute error with the control parameter η to improve the optimization efficiency of the optimization algorithm and the generalization ability of the proposed model. The hyperparameters of the robust ridge regression forecast model are optimized by the PSO algorithm to obtain the optimal hyperparameters. The traffic flow data set is used to train and validate the proposed model. Compared with other optimization methods, the proposed model has the lowest RMSE, MAE, and MAPE. Finally, the traffic flow that forecasted by the proposed model is used to perform anomaly detection. The abnormality of the error between the forecasted value and the actual value is detected by the abnormal traffic flow threshold based on the sliding window. The experimental results verify the validity of the proposed anomaly detection model.

Prediction of chronological age based on Demirjian dental age using robust ridge regression method

Prediction of chronological age based on Demirjian dental age using robust ridge regression method

Robust Ridge regression to solve a multicollinearity and outlier

Regression analysis is one of many methods used for analysing data. Method that used for estimating parameter in linear regression model is ordinary least square (OLS). OLS will give best estimator when all the assumptions are met. But in reality, sometimes not all the assumptions are met. Assumptions that usually violated are multicollinearity and outlier. Ridge regression is a regression method that give constrain on the parameters that used to deal with multicollinearity, meanwhile Robust regression is used to overcome the presence of outlier. Robust regression is a regression method that has robust property that achieved by using S-estimation is used. Ridge regression and Robust regression combined into Robust Ridge regression to overcome multicollinearity and outlier simultaneously.

Robust ridge regression for estimating the effects of correlated gene expressions on phenotypic traits

Statistical packages such as edgeR and DESeq are intended to detect genes that are relevant to phenotypic traits and diseases. A few studies have also modeled the relationships between gene expressions and traits. In the presence of multicollinearity and outliers, which are unavoidable in genetic data, the robust ridge regression estimator can be applied with the trait value as the response variable and the gene expressions as explanatory variables. In some simulation scenarios, the robust ridge estimator is resistant to outliers and less susceptible to multicollinearity than the ordinary least-squares (OLS) estimator. This study investigated the reliability of the robust ridge estimator, in a scenario where the explanatory variables have tail-dependence and negative binomial distributions, by comparing its performance to that of OLS using vine copula to model the tail-dependence among gene expressions. The robust ridge estimator and OLS were both applied to an ecological dataset. First, statistical analysis was used to compare RNA sequencing data between two treatments; then, 15 differentially expressed genes were selected. Next, the regression parameter estimates of robust ridge and OLS for the effects of the 15 contigs (explanatory variables) on trait values (response variables) were compared. Robust ridge regression was found to detect fewer positive and negative slopes than OLS regression. These results indicate that robust ridge regression can be successfully applied for RNA sequencing analysis to estimate the effect of trait-associated genes using real data, and holds great promise as a tool for modeling the association between RNA expression and phenotypic traits.

A Comparative Study between Ridge MM and Ridge Least Trimmed Squares Estimators in Handling Multicollinearity and Outliers

Common problems found in multiple linear regression models are the existence of multicollinearity and outliers. These obstacles usually produce undesirable effects on least squares estimators. Ridge regression estimator is suggested in handling severe multicollinearity while robust estimators such as MM estimator and Least Trimmed Squares (LTS) estimator are recommended in tackling the outlier issues. An even worse scenario is when these two problems occur simultaneously. Combination of both leads to robust ridge regression methods which can be used to handle both conditions simultaneously. In this study, a comparative investigation is carried out to compare the performance between ridge MM and ridge LTS estimators. The Root Mean Square Error (RMSE) and Bias are obtained for each estimator to compare their performances. By using simulation study, Laplace and Cauchy distributions are used in introducing outliers to the simulated data with high multicollinearity ρ = 0.90, 0.95 and 0.98 for sample sizes n=25, 50 and 100. From the results, it is found that Ridge LTS is the best estimator for many combinations of error distributions and degrees of multicollinearity. Similar results were obtained when using two sets of real data.

A comparison of some new and old robust ridge regression estimators

Ridge regression is used to circumvent the problem of multicollinearity among predictors and many estimators for ridge parameter are available in the literature. However, if the level of collinearity among predictors is high, the existing estimators also have high mean square errors (MSE). In this paper, we consider some existing and propose new estimators for the estimation of ridge parameter k. Extensive Monte Carlo simulations as well as a real-life example are used to evaluate the performance of proposed estimators based on the MSE criterion. The results show the superiority of our proposed estimators compared to the existing estimators.

A multi-objective wind speed and wind power prediction interval forecasting using variational modes decomposition based Multi-kernel robust ridge regression

This paper presents a new hybrid multi-objective wind speed and wind power prediction interval forecasting (PIs) model which is the combination of variational mode decomposition (VMD), Multi-kernel robust ridge regression (MKRR) and a multi-objective Chaotic water cycle algorithm (MOCWCA). VMD is applied to decompose the main time series signals into appropriate number of modes that avoids the mutual effects present in between the modes. The VMD based MKRR method is applied to estimate the wind speed and wind power prediction intervals at a prediction interval nominal confidence levels (PINC) of 95%, 90%,85% and 80%, respectively. Further to improve the performance of the proposed prediction model MOCWCA is introduced for the optimization of the prediction models parameters in such a way that multiple objectives are satisfied to produce Pareto-optimal solutions. The wind speed and power data samples for prediction interval forecasting are collected at 30 min and 1 hour time intervals from the Sotavento wind farm located in Spain.

Robust ridge regression based on self-paced learning for multivariate calibration

In this paper, we propose a robust ridge regression model based on self-paced learning (RR-SPL) for the high-dimensional spectroscopic data. The proposed RR-SPL model consists of a weighted least-squares loss term on all training samples, a self-paced regularizer on sample weights, and a smoothness penalty on the model parameter. Designating an explicit form of the self-paced regularizer, the weights that indicate the importance of training samples can be automatically optimized in an augmented ridge regression framework. By increasing the model age, more and more training samples from easy to hard are added into the training set to learn a mature model. As a result, the RR-SPL model can weaken the effect of outliers and obtain an accurate spectra-concentrate relation. Experimental results on simulated data sets and four real near-infrared (NIR) spectra data sets show the effectiveness of the proposed RR-SPL method in a wide range of specific prediction tasks with or without outliers.

An application of robust ridge regression model in the presence of outliers to real data problem

Multicollinearity and outliers are often leads to inconsistent and unreliable parameter estimates in regression analysis. The well-known procedure that is robust to multicollinearity problem is the ridge regression method. This method however is believed are affected by the presence of outlier. The combination of GM-estimation and ridge parameter that is robust towards both problems is on interest in this study. As such, both techniques are employed to investigate the relationship between stock market price and macroeconomic variables in Malaysia due to curiosity of involving the multicollinearity and outlier problem in the data set. There are four macroeconomic factors selected for this study which are Consumer Price Index (CPI), Gross Domestic Product (GDP), Base Lending Rate (BLR) and Money Supply (M1). The results demonstrate that the proposed procedure is able to produce reliable results towards the presence of multicollinearity and outliers in the real data.

Experimental design and data-analysis in label-free quantitative LC/MS proteomics: A tutorial with MSqRob

Label-free shotgun proteomics is routinely used to assess proteomes. However, extracting relevant information from the massive amounts of generated data remains difficult. This tutorial provides a strong foundation on analysis of quantitative proteomics data. We provide key statistical concepts that help researchers to design proteomics experiments and we showcase how to analyze quantitative proteomics data using our recent free and open-source R package MSqRob, which was developed to implement the peptide-level robust ridge regression method for relative protein quantification described by Goeminne et al. MSqRob can handle virtually any experimental proteomics design and outputs proteins ordered by statistical significance. Moreover, its graphical user interface and interactive diagnostic plots provide easy inspection and also detection of anomalies in the data and flaws in the data analysis, allowing deeper assessment of the validity of results and a critical review of the experimental design. Our tutorial discusses interactive preprocessing, data analysis and visualization of label-free MS-based quantitative proteomics experiments with simple and more complex designs. We provide well-documented scripts to run analyses in bash mode on GitHub, enabling the integration of MSqRob in automated pipelines on cluster environments (https://github.com/statOmics/MSqRob). SignificanceThe concepts outlined in this tutorial aid in designing better experiments and analyzing the resulting data more appropriately. The two case studies using the MSqRob graphical user interface will contribute to a wider adaptation of advanced peptide-based models, resulting in higher quality data analysis workflows and more reproducible results in the proteomics community. We also provide well-documented scripts for experienced users that aim at automating MSqRob on cluster environments.

Using Ridge Least Median Squares to Estimate the Parameter by Solving Multicollinearity and Outliers Problems

In the multiple linear regression analysis, the ridge regression estimator is often used to address the problem of multicollinearity. Besides multicollinearity, outliers also constitute a problem in the multiple linear regression analysis. We propose a new biased estimators of the robust ridge regression called the Ridge Least Median Squares (RLMS) estimator in the presence of multicollinearity and outliers. For this purpose, a simulation study is conducted in order to see the difference between the proposed method and the existing methods in terms of their effectiveness measured by the mean square error. In our simulation studies the performance of the proposed method RLMS is examined for different number of observations and the different percentage of outliers. The results of different illustrative cases are presented. This paper also provides the results of the RLMS on a real-life data set. The results show that RLMS is better than the existing methods of Ordinary Least Squares (OLS), Ridge Least Absolute Value (RLAV) and Ridge Regression (RR) in the presence of multicollinearity and outliers.

Alternative ridge robust regression estimator for dealing with collinear influential data points

The multicollinearity in multiple linear regression models and the existence of influential data points are common problems. These problems exert undesirable effects on the least squares estimators. So, it is very important to introduce some alternative biased estimators of the robust ridge regression to overcome the influence of these problems simultaneously. In this paper, alternative biased robust regression estimator is defined by mixing the ridge estimation technique into the robust least median squares estimation to obtain the Ridge Least Median Squares (RLMS). The efficiency of the combined estimator (RLMS) is compared with some existing regression estimators, which namely, the Ordinary Least Squares (LS); Ridge Regression (RR) and Ridge Least Absolute Deviation (RLAD). The numerical results of this study show that, the RLMS regression estimator is more efficient than other estimators, based on, Bias and mean squared error criteria for many combinations of influential data points and degree of multicollinearity. Keywards: Influential Data Points; Multicollinearity; Ridge regression; Ridge Least Absolute Deviation; Ridge Least Median Squares estimation; Bias and Mean Squared Error criteria

Robust ridge regression estimators for nonlinear models with applications to high throughput screening assay data.

Nonlinear regression is often used to evaluate the toxicity of a chemical or a drug by fitting data from a dose-response study. Toxicologists and pharmacologists may draw a conclusion about whether a chemical is toxic by testing the significance of the estimated parameters. However, sometimes the null hypothesis cannot be rejected even though the fit is quite good. One possible reason for such cases is that the estimated standard errors of the parameter estimates are extremely large. In this paper, we propose robust ridge regression estimation procedures for nonlinear models to solve this problem. The asymptotic properties of the proposed estimators are investigated; in particular, their mean squared errors are derived. The performances of the proposed estimators are compared with several standard estimators using simulation studies. The proposed methodology is also illustrated using high throughput screening assay data obtained from the National Toxicology Program.

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.

The efficiency of some robust ridge regression for handling multicollinearity and non-normals errors problems

A common problems in multiple regression models are multicollinearity and non-normal errors, which produce undesirable effects on the least squares estimators. So, it would seem important to combine methods of estimation designed to deal with these problems. In this paper, a comparative investigation was done experimentally for some different estimation methods, which namely the ordinary least squares ( LS) , Ridge Regression ( R ), Ridge least Absolute Deviation (RLAD), Weighted Ridge (WR), Robust M(M) and Robust Ridge regression based on M-estimation (RM). From a simulation study, the resulting robust ridge regression estimator (RM)is efficient than other estimators, using the mean squared error criterion for many combinations of error distribution and degree of multicollinearity.