Levels Of Multicollinearity Research Articles

On some two parameter estimators for the linear regression models with correlated predictors: simulation and application

Regression analysis is widely used to predict the response variable utilizing one or more predictor variables. In many fields of study, the predictors are highly correlated causing multicollinearity problem that severely affects the efficiency of ordinary least square (OLS) estimators by significantly inflating their variances. To solve the multicollinearity problem, various one and two parameter ridge estimators are available in literature. In this article, a class of modified two parameter Lipovetsky–Conklin ridge estimators is proposed based on eigen values of X ′ X matrix that provide an automatic dealing option for treating different levels of multicollinearity. An extensive simulations study followed by real life example is used to evaluate the performance of proposed estimators based on MSE criterion. In most of the simulation conditions, our proposed estimators outperformed the existing estimators.

A new Bayesian ridge estimator for logistic regression in the presence of multicollinearity

This research introduces the Bayesian schemes for estimating logistic regression parameters in the presence of multicollinearity. The Bayesian schemes involve the introduction of a prior together with the likelihood which resulted in the posterior distribution that is not tractable, hence the use of a numerical method i.e Gibbs sampler. Different levels of multicollinearity were chosen to be p = 0.80‚0.85‚0.90‚0.95‚0.99and 0.999to accommodate severe, very severe and nearly perfect state of multicollinearity with sample sizes taken as 10,20,30,50,100,200,300 and 500.Different ridge parameters k were introduced to remedy the effect of multicollinearity .The explanatory variables used were 3 and 7. Model estimation was carried out using Bayesian approach via the Gibbs sampler of Markov Chain Monte Carlo Simulation. The means square error MSE of Bayesian logistic regression estimation was compared with the frequentist methods of the estimation. The result shows a minimum mean square error with the Bayesian scheme compared to the frequentist method.

A Monte Carlo Experiment on the Asymptotic Efficiency of System Estimators under Multicollinearity

Classical linear regression model assumes that there is no multicollinearity among the explanatory variables in a regression model. Contrary to this assumption, where multicollinearity is perfect, the regression coefficients of the explanatory variables are indeterminable and their standard errors are infinite. On the other hand, where multicollinearity is less than perfect, the regression coefficients, although determinable, possess large standard errors. This implies that the coefficients cannot be estimated with great precision. Hence multicollinearity problem is a major problem in econometric analysis. Using Monte Carlo Simulation, we evaluated the asymptotic efficiency of six estimators (OLS, ILS,2SLS, 3SLS, LIML and FIML), under different magnitudes of the unintended linear relationship between the exogenous variables. Using the SSR criteria, we found that OLS followed by ILS turned out the best estimates amongst the six estimators under multicollinearity. We also found that with increasing sample size, there is no remarkable asymptotic effect in the performance of the estimators at the levels of multicollinearity.

Comparison of Least Squares and Principal Components Regression Analysis

The aim of this study is to compare Least Squares Regression (LSR) and Principal Components Regression (PCR) results when multicollinearity is determined in a dataset.In order to examine the effect of the degree of multicollinearity in the study, 10 datasets with different levels of multicollinearity were derived. Each data set consists of three independent and one dependent variable, and the variables were derived from the standard normal distribution. The multicollinearity status in the derived data has been demonstrated by the commonly used metrics to determine multicollinearity. Least Squares and Principal Components Regression was applied to all datasets.
 When generating multicollinearity, all relationships were defined as positive in data simulation. However, the sign of the regression coefficients for the second (X2) and third (X3) independent variables were obtained as reverse (negative) as one of the expected effects of multicollinearity in Least Squares analysis. In the analysis of the Principal Components Regression, the sign of coefficients was found to be in the right direction (positive). The sign of the coefficients obtained from OLS and PCR were different and they also differed in magnitude. In addition, the standard errors of the coefficients in PCR results were lower than OLS results.
 The existence of multicollinearity must be examined while performing multiple linear regression analysis, and if multicollinearity is determined, one of the methods that can solve this situation should be used. Otherwise, the estimations to be made as a result of regression may lead to wrong results. In line with the results of this study, it is recommended to use Principal Components Regression instead of Least Squares regression in case of multicollinearity in the data.

Comparison of fast regression algorithms in large datasets

The aim is to compare the performances of fast regression methods, namely dimensional reduction of correlation matrix (DRCM), nonparametric dimensional reduction of correlation matrix (N-DRCM), variance inflation factor (VIF) regression, and robust VIF (R-VIF) regression in the presence of multicollinearity and outliers problems. In all simulation-scenarios, all the target variables were chosen for final models using four methods. The DRCM and N-DRCM are the methods that reach the final model in the shortest time, respectively. The time to reach the final model using R-VIF regression was approximately twice shorter than that of VIF regression. In each method, as the number of variables and the level of outliers increased, the time taken to reach the final model increased. When the level of multicollinearity and the number of variables (p > 500) increased, the times to reach the final models using DRCM in datasets with outliers were slightly shorter than the those of N-DRCM. The largest numbers of noise variables were selected to the model using DRCM and N-DRCM, but the least number of them were selected to the model using the R-VIF regression. The RMSE values obtained using DRCM, N-DRCM and VIF regression were similar in each scenario. As a result of the real dataset, the final model selected using R-VIF regression had the highest R2. It also had the lowest RMSE value among those obtained with other approaches excluding VIF regression. As such, the R-VIF regression method demonstrated a better performance than the others in all datasets.

A Monte Carlo Study for Dealing with Multicollinearity and Autocorrelation Problems in Linear Regression Using Two Stage Ridge Regression Method

In the multiple linear regression model, the problem of multicollinearity may come together with autocorrelation; therefore, several methods of estimation are developed to deal with this case; Two-Stage Ridge Regression (TR) is one of them. This article's main objective is to run a Monte Carlo simulation to investigate the impact of both problems, Multicollinearity and Autocorrelation, in multiple linear regression model on the performance of the TR. The simulation is carried out under different levels of multicollinearity, and sets of autocorrelation coefficient, taking into account different sample sizes. Some new properties for the TR method, including expectation, variance and mean square error, are droved. In contrast, the study also has developed some techniques to estimate the biasing parameter for the TR by modifying some popular techniques used in ridge regression (RR). Moreover, Mean Square Error is used as a base for evaluation and comparison. The empirical findings from the simulations revealed that the TR estimator performs better than the RR, and the values of the biasing parameter under the TR are always less than that under the RR. This paper contributes to the existing literature on developing new estimation methods used to overcome the presence of mixed problems in a linear regression model and studying their properties.

Tobit Liu estimation of censored regression model: an application to Mroz data and a Monte Carlo simulation study

Multicollinearity is often ignored in censored regression model. In this respect, Tobit Liu estimation is proposed as an alternative in order to dispel the adverse effects of multicollinearity on the maximum likelihood estimation. Tobit Liu estimator is compared with Tobit ridge estimator and Tobit maximum likelihood estimator, theoretically via mean square error. For empirical analysis, we prefer Mroz dataset which is a phenomenon in the context of censored regression. Since many studies have investigated Mroz dataset by neglecting multicollinearity, we handle this dataset in the context of multicollinearity in this paper. Moreover, it is seen that multicollinearity is not included in pioneer studies' simulation designs. Therefore, we conduct a Monte Carlo simulation study where we take account of different levels of multicollinearity while generating data. Briefly, this study contributes to filling an important gap in the literature and proposes to investigate other econometrically censored data in the context of multicollinearity.

Generalized ridge estimators adapted in structural equation models

Multicollinearity is detected via regression models, where independent variables are strongly correlated. Since they entail linear relations between observed or latent variables, the structural equation models (SEM) are subject to the multicollinearity effect, whose numerous consequences include the singularity between the inverse matrices used in estimation methods. Given to this behavior, it is natural to understand that the suitability of these estimators to structural equation models show the same features, either in the simulation results that validate the estimators in different multicollinearity degrees, or in their application to real data. Due to the multicollinearity overview arose from the fact that the matrices inversion is impracticable, the usage of numerical procedures demanded by the maximum likelihood methods leads to numerical singularity problems. An alternative could be the use of the Partial Least Squares (PLS) method, however, it is demanded that the observed variables are built by assuming a positive correlation with the latent variable. Thus, theoretically, it is expected that the load signals are positive, however, there are no restrictions to these signals in the algorithms used in the PLS method. This fact implies in corrective areas, such as the observed variables removal or new formulations of the theoretical model. In view of this problem, this paper aimed to propose adaptations of six generalized ridge estimators as alternative methods to estimate SEM parameters. The conclusion is that the evaluated estimators presented the same performance in terms of accuracy, precision while considering the scenarios represented by model without specification error and model with specification error, different levels of multicollinearity and sample sizes.

A Comparative Analysis on Some Estimators of Parameters of Linear Regression Models in Presence of Multicollinearity

Linear regression is the measure of relationship between two or more variables known as dependent and independent variables. Classical least squares method for estimating regression models consist of minimising the sum of the squared residuals. Among the assumptions of Ordinary least squares method (OLS) is that there is no correlations (multicollinearity) between the independent variables. Violation of this assumptions arises most often in regression analysis and can lead to inefficiency of the least square method. This study, therefore, determined the efficient estimator between Least Absolute Deviation (LAD) and Weighted Least Square (WLS) in multiple linear regression models at different levels of multicollinearity in the explanatory variables. Simulation techniques were conducted using R Statistical software, to investigate the performance of the two estimators under violation of assumptions of lack of multicollinearity. Their performances were compared at different sample sizes. Finite properties of estimators’ criteria namely, mean absolute error, absolute bias and mean squared error were used for comparing the methods. The best estimator was selected based on minimum value of these criteria at a specified level of multicollinearity and sample size. The results showed that, LAD was the best at different levels of multicollinearity and was recommended as alternative to OLS under this condition. The performances of the two estimators decreased when the levels of multicollinearity was increased.

Improving the validity of theory testing in logistics research using correlated components regression

ABSTRACTThe purpose of this logistics research methods article is to empirically test and introduce correlated components regression (CCR) as a new statistical technique that will improve the accuracy and validity in testing logistics theoretical models and hypothesised relationships. The purpose of the current study is to use CCR analysis as technique to address multicollinearity. Customer satisfaction data with parcel carriers is analysed with using CCR and multiple regression. To determine the best regression model of these two approaches, cross-validation R2 values are used. In addition, comparisons are made to examine the standardised beta coefficients from both methods and to assess the possible impact from high levels of multicollinearity. Findings of the analysis suggest that CCR has a significantly higher cross-validation R2 value and thus is determined the best model of these two approaches.

Combined Parameters Estimation Methods of Linear Regression Model with Multicollinearity and Autocorrelation

Multicollinearity and autocorrelation are two major problems often encounter in regression analysis. Estimators for their separate investigation have been developed even though they are not without challenges. However, the two problems occasionally do occur together. In this paper effort is made to provide some combined estimators based on Feasible Generalized Linear Estimator (CORC and ML) and Principal Components (PCs) Estimator that estimate the parameters of linear regression model when the two problems are in the model. A linear regression model with three explanatory variables distributed normally and uniformly as well as exhibiting multicollinearity and autocorrelation was considered through Monte Carlo experiments at four levels of sample size .The experiments were conducted and the performances of the various proposed combined estimators with their separate ones and the Ridge estimator were examined and compared using the Mean Square Error (MSE) criterion by ranking their performances at each level of multicollinearity, autocorrelation and parameter. The ranks were further summed over the number of parameters. Results show that the proposed estimator MLPC1 is generally best even though the CORCPC1 and PC1 often compete favorably with it. Moreover with increased sample size, the CORCPC12 and MLPC12 are often best.

English

Teachers have been widely acknowledged to critically influence students’ learning and achievement. Nonetheless, the effects of teacher characteristics on student communication proficiency in the context of English as foreign language (EFL) learning have not received adequate attention among scholars. In contrast to most studies in the area, this article focuses on the influence of teacher-student interpersonal engagements, namely teacher’s personality and monitoring, on EFL student communication proficiency. The purpose of this study is to explore whether teacher’s characteristics and environmental learning factors influence students’ overall communication proficiency. Drawing on sociocultural theory, the authors assume that EFL learning is bi-directional in nature. In addition to conceptualizing the direct impact of domain-specific determinants of communication proficiency, internal classroom conditions and external college facilities were assessed for possible moderating effects. Using a sample of college students in twelve different campus locations from Institut Kemahiran Mara (IKM), across Malaysia, the authors performed a regression analysis to empirically test the proposed research hypotheses. All constructs demonstrated low levels of multicollinearity and measurement scales indicated sufficient reliability and validity. Results show that teacher’s personality is an important determinant of student proficiency in English. Monitoring efforts, however, did not have a direct influence on English proficiency but coupled with teacher’s personality and college facilities, respectively, the coefficient interaction effects became significant. The results also reveal that classroom conditions do not interact with teacher’s personality to improve student learning outcomes, nonetheless acts more predominantly as a predictor that directly enhances students’ learning. Important research implications and future research directions are suggested. DOI: http://dx.doi.org/10.17576/GEMA-2014-1401-07

Empirical Determination of the Tolerable Sample Size for Ols Estimator in the Presence of Multicollinearity (<i>ρ</i>)

This paper investigates the tolerable sample size needed for Ordinary Least Square (OLS) Estimator to be used when there is presence of Multicollinearity among the exogenous variables of a linear regression model. A regression model with constant term (β0) and two independent variables (with β1 and β2 as their respective regression coefficients) that exhibit multicollinearity was considered. A Monte Carlo study of 1000 trials was conducted at eight levels of multicollinearity (0, 0.25, 0.5, 0.7, 0.75, 0.8, 0.9 and 0.99) and sample sizes (10, 20, 40, 80, 100, 150, 250 and 500). At each specification, the true regression coefficients were set at unity while 1.5, 2.0 and 2.5 were taken as the hypothesized value. The power value rate was obtained at every multicollinearity level for the aforementioned sample sizes. Therefore, whether the hypothesized values highly depart from the true values or not once the multicollinearity level is very high (i.e. 0.99), the sample size needed to work with in order to have an error free estimation or the inference result must be greater than five hundred.

A New Ridge Regression Causality Test in the Presence of Multicollinearity

The VAR lag structure applied for the traditional Granger causality (GC) test is always severely affected by multicollinearity due to autocorrelation among the lags. Therefore, as a remedy to this problem we introduce a new Ridge Regression Granger Causality (RRGC) test, which is compared to the GC test by means of Monte Carlo simulations. Based on the simulation study we conclude that the traditional OLS version of the GC test over-rejects the true null hypothesis when there are relatively high (but empirically normal) levels of multicollinearity, while the new RRGC test will remedy or substantially decrease this problem.

Two-pass estimation of risk premiums with multicollinear and near-invariant betas

Abstract This paper investigates the reliability of the two-pass (TP) estimators of factor risk prices when betas (multifactor loadings) have high levels of cross-sectional correlation (multicollinearity) and/or when some of them have small cross-sectional variations (near-invariance). Our simulation results show the following. First, the TP estimators can have biases larger than 100% of true risk prices when data are generated by the betas with high levels of multicollinearity and invariance that can be observed from actual data. Second, the t-tests for hypotheses related to risk prices and pricing intercepts have only limited power. The levels of multicollinearity and invariance of betas can vary depending on the assets and sample periods used in estimation. Thus, we propose use of two pre-diagnostic statistics to measure these levels. Many previous studies have investigated the finite-sample properties of the TP estimators using the data generated with the estimated betas from actual data. Our results indicate that simulation outcomes can lead to quite different conclusions, depending on the levels of multicollinearity and invariance of the betas used to generate the data.

Volume equations for poplars growing on farmland in Sweden

Volume equations for poplars growing on farmland in Sweden. The aim of the study was to construct a simple volume equation for poplar and to evaluate the equation together with seven published equations. Data from 72 poplars were collected from 37 stands in central and southern Sweden (lat. 55–60°N). The stands’ mean age and density was 21 years and 940 stems ha−1, respectively and the trees’ mean diameter at breast height (DBH) was 25 cm. One simple volume equation with three parameters to be estimated for poplar was constructed with DBH and total height (H) as independent variables. This equation was analyzed and compared to seven published simple equations. The best performance, in terms of low values of Root Mean Square Error (RMSE) and Absolute Bias, was achieved by the equations developed by Eriksson and by Scott. However, due to these equations high level of multicollinearity an alternative recommendation was made where the constructed equation is recommended.

MENSURAÇÃO COM INDICADORES FORMATIVOS NAS PESQUISAS EM ADMINISTRAÇÃO DE EMPRESAS: COMO LIDAR COM A MULTICOLINEARIDADE ENTRE ELES?

<p>O uso de indicadores formativos em modelos de equações estruturais é um dos motivos para se usar o Partial Least Squares Path Modeling (PLSPM) em vez do LISREL. Além disso, tem aumentado o uso de PLS-PM nas pesquisas em Administração de Empresas. O objetivo desta pesquisa é a avaliação dos efeitos da multicolinearidade de indicadores formativos sobre os valores estimados para os pesos fatoriais e para o coeficiente estrutural. Para simular diferentes valores de multicolinearidade varia-se a quantidade de indicadores por variável latente, bem como a correlação entre eles, sendo gerados 540 modelos com diferentes valores para os<br />coeficientes estruturais. Como conclusão, verifica-se que o aumento da variabilidade dos pesos fatoriais é incrementado para valores menores do coeficiente estrutural. Apesar da impossibilidade de se avaliar a importância relativa de cada indicador para a mensuração do construto, sob a influência da multicolinearidade, observa-se que os coeficientes estruturais não são alterados. Também se observa a inconsistência do PLS-PM quando se usa menos de cinco indicadores por variável latente (consistency at large) e quando a confiabilidade é inferior a 0,9. Ao final são feitas recomendações para a minimização dos efeitos da<br />multicolinearidade e sugerem-se direções para novas pesquisas.<br /><br /></p>

EFFECT OF CORRELATIONS AND EQUATION IDENTIFICATION STATUS ON ESTIMATORS OF A SYSTEM OF SIMULTANEOUS EQUATION MODEL

In simultaneous equations model, multicollinearity and status of identification of the equations have been observed to influence estimation of the model parameters. The error terms of each equation in the model are also expected to be correlated with each other. This study therefore examined the effect of multicollinearity, correlation between error terms and status of identification of equations on six methods of parameter estimation in a simultaneous equations model using Monte Carlo approach. A two equation model, with one equation exactly identified and the other over identified, was considered. The levels of multicollinearity among the exogeneous variables were specified as r = 0.3, 0.6, 0.8, 0.9 and 0.99 and that of correlation between error terms as l = 0.3, 0.6, and 0.9. A Monte Carlo experiment of 250 trials was carried out at three sample sizes (20, 50 and 100). The six estimation methods; Ordinary Least Squares (OLS), Indirect Least Squares (ILS), Limited Information Maximum Likelihood (LIML), Two Stage Least Squares (2SLS), Full Information Maximum Likelihood (FIML) and Three Stage Least Squares (3SLS); were ranked according to their performances. Finite properties of estimators’ criteria namely bias, absolute bias, variance and mean squared error were used for comparing the methods. An estimator is best at a specified level of multicollinearity, correlation between error terms and sample size if it has minimum total rank over the model parameters and the criteria. Results show that the OLS estimator is best in estimating the parameters of the exactly identified equation at severe level of multicollinearity ( r ® 1) at all sample sizes. At other levels of multicollinearity, the best estimator is FIML or 3SLS except when the correlation between error terms is low ( l = 0.3). At this instance, the best estimators are LIML and 2SLS. The parameters of over identification model are best estimated with FIML or 3SLS at all levels of multicollinearity, correlation between error terms and at all sample sizes.

Effect of Multicollinearity on Power Rates of the Ordinary Least Squares Estimators

Inferences on the parameter estimates of Ordinary Least Square (OLS) estimator in regression model when regressors exhibit multicollinearity is a problem in that large standard errors of the regression coefficients which cause low t-statistic value often result into the acceptance of the null hypothesis. This paper, therefore, makes efforts to investigate the effect of multicollinearity on the power rates of the OLS estimator. A regression model with constant term (β0) and two independent variables (with (β1 and (β2 as their respective regression coefficients) that exhibit multicollinearity was considered. A Monte Carlo study of 1000 trials was conducted at eight levels of multicollinearity (0, 0.25, 0.5, 0.7, 0.75, 0.8, 0.9 and 0.99) and sample sizes (10, 20, 40, 80, 100, 150, 250 and 500). At each specification, the true regression coefficients were set at unity while 1.5, 2.0 and 2.5 were taken as their hypothesized values. Results show that at each hypothesized value of β0 the power rate is the same at all the levels of multicollinearity at a specified sample size and that the error rate decreases asymptotically. Furthermore as the hypothesized value increases, results do not only show that the power rate increases but tends faster to one asymptotically. The pattern of effect of power rate of β1 and β2 is the same as that of β0 except that at each hypothesized value the power rate decreases as level of multicollinearity increases at a specified sample size. Consequently, increasing the sample size increase the power rate of the OLS estimator in all the levels of multicollinearity.

Multicollinearity and Modular Maximum Entropy Leuven Estimator

A high degree of multicollinearity often has a detrimental effects on the estimation of a linear econometric (regression) model due to an intricate internecine sharing among the estimated regression coefficients (beta). Paris (2001) introduced the Maximum Entropy Leuven (MEL) estimator. It exploits the information available in the sample data more efficiently than the OLS does, and unlike GME estimator, it does not require any additional information to be supplied by the researcher. Paris concludes: under any level of multicollinearity, MEL estimator uniformly dominates the OLS estimator according to the mean squared error criterion. It rivals also the GME estimator without requiring any subjective additional information. Paris used the Euclidean norm to obtain prob(beta). In this paper we obtain prob(beta) using the absolute norm of beta and investigate into its effects on the performance of the Maximum Entropy estimator. We obtain a new estimator of regression coefficients by solving the reformulated problem. This new estimator is not fully a la Paris (2001) and hence we name it as the Modular Maximum Entropy Leuven (MMEL) estimator. Monte Carlo study has been conducted to compare the performance of the MEL estimator (Paris) and the new MMEL estimator. We observe that overall, the performance of MMEL (in terms of mean estimated coefficients as well as the RMS values) is much superior to that of the MEL estimator. It is pertinent to note that obtaining prob(beta) is the most crucial task before the scientist if he chooses to use the maximum entropy estimator (MEL, MMEL or any variant thereof). After all, the mathematics of probability suggests us that given a sample description space S, probability is a function which assigns a non-negative real number to every event A, denoted by P(A) and it is called the probability of the event A. The probability function is defined on a Borel field of events conformal to the axioms of positiveness, certainty and union. Under these axioms, there could be several different rules of assignment, ranging from subjective judgement backed up by a rational belief (JM Keynes) to counting the number of success in the repeated trials. In our study MEL does this assignment in the one way and the MMEL does that in the other way. There could be many more (possibly better) rules of assignment of probability. Thus, the subjective (or exogenous) element that was explicit in Golan et al. reappears in the MEL family of estimators, although in another garb.