Modified Ridge Regression Research Articles

Ridge Estimation's Effectiveness for Multiple Linear Regression with Multicollinearity: An Investigation Using Monte-Carlo Simulations

The goal of this research is to compare multiple linear regression coefficient estimations with multicollinearity. In order to quantify the effectiveness of estimations by the mean of average mean square error, the ordinary least squares technique (OLS), modified ridge regression method (MRR), and generalized Liu-Kejian method (LKM) are compared (AMSE). For this study, the simulation scenarios are 3 and 5 independent variables with zero mean normally distributed random error of variance 1, 5, and 10, three correlation coefficient levels; i.e., low (0.2), medium (0.5), and high (0.8) are determined for independent variables, and all combinations are performed with sample sizes 15, 55, and 95 by Monte Carlo simulation technique for 1,000 times in total. As the sample size rose, the AMSE decreased. The MRR and LKM both outperformed the LSM. At random error of variance 10, the MRR is the most suitable for all circumstances.

Measuring Local Influential Observations in Modified Ridge Regression

Measuring Local Influential Observations in Modified Ridge Regression

Influenza Prediction: Analyzing Machine Learning Algorithms

Analyzing online or digital data for detecting epidemics is one of the hot areas of research and now becomes more relevant during the present outbreak of Covid-19. There are several different types of the influenza virus and moreover they keep evolving constantly in the same manner the COVID-19 virus has done. As a result, they pose a greater challenge when it comes to analyzing them, predicting when, where and at what degree of severity it will outbreak during the flu season across the world. There is need for greater surveillance to both seasonal and pandemic influenza to ensure the health and safety of the mankind. The objective of work is to apply machine learning algorithms for building predictive models that can predict where the occurrence, peak and severity of influenza in each season. For this work we have considered a freely available dataset of Ireland which is recorded for the duration of 2005 to 2016. Specifically, we have tested three ML Algorithms namely Linear Regression, Support Vector Regression and Random Forests. We found Random Forests is giving better predictive results. We also conducted experiment through weka tool and tested Zero R, Linear Regression, Lazy Kstar, Random Forest, REP Tree, Multilayer Perceptron models. We again found the Random Forest is performing better in comparison to all other models. We also evaluated other regression models including Ridge Regression, modified Ridge regression, Lasso Regression, K Neighbor Regression and evaluated the mean absolute errors. We found that modified Ridge regression is producing minimum error. The proposed work is inclined towards finding the suitability & appropriate ML algorithm for solving this problem on Flu.

Influential diagnostics with Pena’s statistic for the modified ridge regression

In a linear regression model, the estimation of regression parameters is affected by some anomalous observations in the data set. Thus, identification of these observations is one of the essential steps in regression analysis. Detection of these influential observations becomes more complex when serious multicollinearity among regressors is also observed. In the present article, we formulated Pena’s statistic for each point while considering the modified ridge regression (MRR) estimator. Using this statistic, we showed that when MRR was used to mitigate the effects of multicollinearity, the influence of some observations could be significantly changed. The normality of this statistic was also discussed and it was proved that it could detect a subset of high modified ridge leverage outliers. The Monte Carlo simulations were used for empirical results and an example of real data was presented for illustration.

Influence in the Elliptical Modified Ridge Regression

In this article, we present diagnostic methods for the modified ridge regression under elliptical model based on the pseudo-likelihood function. The maximum likelihood estimators of the parameters in the modified ridge elliptical model are given and local influence measures are developed. Finally, illustration of our methodology is given through a numerical example.

Local Influence Diagnostics in the Modified Ridge Regression

Calculations of local influence curvatures have been well developed when the ordinary least squares method is applied. In this article, we discuss the assessment of local influence under the modified ridge regression. Using a pseudo-likelihood function, we express the normal curvatures of local influence for three useful perturbation schemes in interpretable forms. Two illustrative examples are analyzed by the methodology developed in the article.

Detection of influential observations in ridge regression and modified ridge regression

The detection of influential observations is important because of their unduly large influence on the regression analysis results. Numerous diagnostics on identifying these observations are developed in the regression analysis. Pena's statistic is one of the proposed diagnostics. In this study, Pena's approach is formulated to ridge regression (RR) and modified ridge regression (MRR). The real data and artificial data are used to see performance of proposed diagnostic for the RR and MRR on detecting influential observations.

More on the restricted ridge regression estimation

Several alternative methods for derivation of the restricted ridge regression estimator (RRRE) are provided. Theoretical comparison and relationship of RRRE with related methods for regression with the multicollinearity problem are described. We also find inter-connections among RRRE, ordinary ridge regression estimator (ORRE), restricted least squares estimator (RLSE), modified ridge regression estimator (MRRE) and restricted modified generalized ridge estimator (RMGRE). Finally, numerical comparison, in addition to theoretical derivation, is also conducted with a Monte Carlo simulation and a real data example.

Assessing global influential observations in modified ridge regression

We occasionally find that a small subset of the data exerts a disproportionate influence on the fitted regression model. That is, parameter estimates or predictions may depend more on the influential subset than on the majority of the data. We would like to locate these influential points and assess their impact on the model. If these influential points are bad values then they should be eliminated. On the other hand, there may be nothing wrong with these points, but if they control key model properties, as we would like for them to, they could affect the use of the model. When modified ridge regression (MRR) is used to mitigate the effects of multicollinearity, the influence of observations can be drastically modified. In this paper, we propose a case deletion formula to detect influential points in MRR. The [Longley, J.W., 1967. An appraisal of least squares programs for electronic computers from the point of view of the user. Journal of American Statistical Association 62, 819–841] data is used to illustrate our methodology.

Confounder Selection in Environmental Epidemiology: Assessment of Health Effects of Prenatal Mercury Exposure

The purpose of the study is to compare different approaches to the identification of confounders needed for analyzing observational data. Whereas standard analysis usually is conducted as if the confounders were known a priori, selection uncertainty also must be taken into account. Confounders were selected by using backward elimination (BE), change in estimate (CIE) method, Akaike information criterion, Bayesian information criterion (BIC), and an empirical approach using a priori information. A modified ridge regression estimator, which shrinks effects of confounders toward zero, also was considered. For each criterion, uncertainty in the estimated exposure effect was assessed by using bootstrap simulations for which confounders were selected in each sample. These methods were illustrated by using data for mercury neurotoxicity in Faroe Islands children. Point estimates and standard errors of mercury effects on confounder-sensitive neurobehavioral outcomes were calculated for each selection procedure. The full model and the empirical a priori model showed approximately the same precision, and these methods were (slightly) inferior to only modified ridge regression. Lower precisions were obtained by using BE with a low cutoff level, BIC, and CIE. Standard analysis ignores model selection uncertainty and is likely to yield overoptimistic inferences. Thus, the traditional BE procedure with p = 5% should be avoided. If data-dependent procedures are required for confounder identification, we recommend that inferences be based on bootstrap statistics to describe the selection process.

Mean squared error comparisons of the modified ridge regression estimator and iiie restricted ridge regression estimator

Swindel (1976) introduced a modified ridge regression estimator based on prior information. Sarkar (1992) suggested a new estimator by combining in a particular way the two approaches followed in obtaining the restricted ieast squares and ordinary ndge regression estimators. In this paper we compare the mean square error matrices of the modified ridge regression estimator based on prior information and die restricted ridge regression estimator introduced by Sarkar (1992). We stated a sufficient condition for the mean square error matrix of the modified ndge regression estimator to exceed the mean square enor matrix of the restricted ridge regression estimator.

Length modified ridge regression

Biased regression methods may improve considerably on ordinary least squares regression with few or noisy data, or when the predictor variables are highly collinear. In the present work, I present a new, biased method that modifies the ordinary least squares estimate by adjusting each element of the estimated coefficient vector. The adjusting factors are found by minimizing a measure of prediction error. However, the optimal adjusting factors depend on the unknown coefficient vector as well as the variance of the noise, so in practice these are replaced by preliminary estimates. The final estimate of the coefficient vector has the same direction as the preliminary estimate, but the length is modified. Ridge regression is used as the principal method to find the preliminary estimate, and the method is called length modified ridge regression. In addition, length modified principal components regression is considered. The prediction performance of the methods are compared to other regression methods (ridge, James-Stein, partial least squares, principal components and variable subset selection) in a simulation study. Of all methods considered, length modified ridge regression shows the overall best behaviour. The improvement over ridge regression is moderate, but significant, especially when the data are few and noisy.

Continuum Regression is Not Always Continuous

SUMMARY Continuum regression (CR) is regression of a response y on that linear combination t γ of explanatory variables which maximizes r 2(y, t) var(t)γ, where γ is chosen from a continuum of candidates. CR includes several common methods as special cases: ordinary least squares, principal component regression, partial least squares and a modified ridge regression. We demonstrate that CR, despite its name, can yield a predictor that is a discontinuous function of γ and of the calibration data. We illustrate this with a set of real data. The relationship between CR and ridge regression is the key to understanding this phenomenon.

Some remarks on a ridge-type-estimator and good prior means

Pliskin (1987) compared modified ridge regression estimators based on prior information with respect to their mean square error matrices. A further characterization of good prior mean is given here, and the case of different ridge parameters is also considered.

A ridge-type estimator and good prior means

Swindel (1976) introduced a modified ridge regression estimator based on prior information. A necessary and sufficient condition is derived for Swindel's proposed estimator to have lower risk than the conventional ordinary ridge regression estimator when both estimators are computed using the same value of k.