Positive-part Stein-rule Estimator Research Articles

MSE performance of the weighted average estimators consisting of shrinkage estimators

ABSTRACTIn this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.

Risk Comparison of Improved Estimators in a Linear Regression Model with MultivariatetErrors under Balanced Loss Function

Under a balanced loss function, we derive the explicit formulae of the risk of the Stein-rule (SR) estimator, the positive-part Stein-rule (PSR) estimator, the feasible minimum mean squared error (FMMSE) estimator, and the adjusted feasible minimum mean squared error (AFMMSE) estimator in a linear regression model with multivariateterrors. The results show that the PSR estimator dominates the SR estimator under the balanced loss and multivariateterrors. Also, our numerical results show that these estimators dominate the ordinary least squares (OLS) estimator when the weight of precision of estimation is larger than about half, and vice versa. Furthermore, the AFMMSE estimator dominates the PSR estimator in certain occasions.

A note on the properties of Stein-rule and inequality restricted estimators when the regression model is over-fitted

This paper considers the effect of an erroneous inclusion of regressors on the risk properties of the Stein-rule, positive-part Stein-rule and inequality restricted and pre-test estimators in a linear regression model. The two Stein-rule estimators are considered when extraneous information is available in the form of a set of multiple equality constraints on the coefficients, while the inequality estimators are considered under the case of a single inequality constraint. It is shown that the inclusion of wrong regressors has only minimal effect on the properties of the Stein-rule and positive-part Stein-rule estimators, and no effect at all on the inequality restricted and pre-test estimators when there is a single inequality constraint.

Robustness of Stein-type estimators under a non-scalar error covariance structure

The Stein-rule (SR) and positive-part Stein-rule (PSR) estimators are two popular shrinkage techniques used in linear regression, yet very little is known about the robustness of these estimators to the disturbances’ deviation from the white noise assumption. Recent studies have shown that the OLS estimator is quite robust, but whether this is so for the SR and PSR estimators is less clear as these estimators also depend on the F statistic which is highly susceptible to covariance misspecification. This study attempts to evaluate the effects of misspecifying the disturbances as white noise on the SR and PSR estimators by a sensitivity analysis. Sensitivity statistics of the SR and PSR estimators are derived and their properties are analyzed. We find that the sensitivity statistics of these estimators exhibit very similar properties and both estimators are extremely robust to MA(1) disturbances and reasonably robust to AR(1) disturbances except for the cases of severe autocorrelation. The results are useful in light of the rising interest of the SR and PSR techniques in the applied literature.

PMSE performance of the Stein-rule and positive-part Stein-rule estimators in a regression model with or without proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of the predictive mean squared error (PMSE) of the Stein-rule (SR) estimator and the positive-part Stein-rule (PSR) estimator for the regression coefficients when the proxy variables are used. We examine the effect of using the proxy variables on the risk performances of the SR and PSR estimators. It is shown analytically that the PSR estimator dominates the SR estimator even when the proxy variables are used. Also, our numerical results show that using the proxy variables is preferable to omitting the relevant regressors.

PMSE PERFORMANCE OF THE BIASED ESTIMATORS IN A LINEAR REGRESSION MODEL WHEN RELEVANT REGRESSORS ARE OMITTED

In this paper, we consider a linear regression model when relevant regressors are omitted. We derive the explicit formulae for the predictive mean squared errors (PMSEs) of the Stein-rule (SR) estimator, the positive-part Stein-rule (PSR) estimator, the minimum mean squared error (MMSE) estimator, and the adjusted minimum mean squared error (AMMSE) estimator. It is shown analytically that the PSR estimator dominates the SR estimator in terms of PMSE even when there are omitted relevant regressors. Also, our numerical results show that the PSR estimator and the AMMSE estimator have much smaller PMSEs than the ordinary least squares estimator even when the relevant regressors are omitted.

Pre-test double k-class estimators in linear regression

Abstract In this paper, we consider pre-test double k-class (PTKK) estimators for coefficients in a linear regression model. The exact formulas of predictive mean-squared error (PMSE) of PTKK estimators are derived, and sufficient conditions for the PTKK estimator to dominate the double k-class estimator are shown. Some examples are given, and the PMSE performances of the positive-part Stein-rule estimator, the adjusted minimum mean squared error estimator, and the PTKK estimator with an ad hoc parameter value are compared by numerical evaluations.

Improved biased estimation in an ANOVA model

The problem of simultaneous estimation of means in one-way analysis of variance (ANOVA) is considered when it is suspected that the mean parameters are equal, but with some degree of uncertainty. Improved estimators using Stein-rule and preliminary test approach are proposed. The explicit forms of the biases and risk functions of the proposed estimators are derived. The relative performance of the positive-part Stein-rule estimator (PSE) and the usual Stein-rule estimator (SE) is critically assessed with the aid of the quadratic loss function. It is shown analytically as well as computationally that PSE dominates the classical SE in the whole parameter space. Also the improved preliminary test estimator (IPE) is shown to have smaller risk than that of the classical PE in the entire parameter space.

MSE performance of a heterogeneous pre-test estimator

In this paper, we consider a linear regression model and examine the MSE performance of a heterogeneous pre-test estimator whose components are the adjusted minimum mean-squared error (AMMSE) estimator and the Stein-rule (SR) estimator. It is shown that the heterogeneous pre-test estimator dominates the SR estimator if a critical value of the pre-test is chosen appropriately. By numerical evaluations it is shown that if the number of independent variables (say, k) is 3 and the critical value of the pre-test is chosen appropriately, then the heterogeneous pre-test estimator dominates the positive-part Stein-rule (PSR) estimator. Also, when k = 4 and 5, the MSE performances of the heterogeneous pre-test and PSR estimators are comparable.

An MSE comparison of the restricted Stein-rule and minimum mean squared error estimators in regression

In this paper, we consider the feasible minimum mean squared error (FMMSE) estimator and the adjusted FMMSE (AFMMSE) estimator which are obtained by shrinking the ordinary least squares (OLS) estimator towards the restricted least squares estimator. We derive the formulas of MSE for the restricted FMMSE and AFMMSE estimators. By numerical evaluations, the MSE performances of the restricted FMMSE and AFMMSE estimators are compared with that of the restricted positive-part Stein-rule estimator.

Mes performance of the minimum mean squared error estimators in a linear regression model when relevant regressors are omitted

In this paper, we consider a linear regression model when relevant regressors are omitted. We derive the explicit formulae of the mean squared errors (MSE's) of the feasible minimum MSE (FMMSE) estimator and the adjusted FMMSE (AFMMSE) estimator. By numerical evaluations, we compare the MSE performances of the FMMSE and AFMMSE estimators with those of the Stein-rule (SR) and positive-part Stein-rule (PSR) estimators. It is shown that when there are omitted regressors, the MSE performances of the AFMMSE and PSR estimators are comparable when the number of regressors included in the specified model (sayk 1) is larger than or equal to 8, and the MSE performance of the AFMMSE estimator is better than that of the PSR estimator when k 1≤5 and the model misspecification is severe.

The exact general formulae for the moments and the MSE dominance of the Stein-rule and positive-part Stein-rule estimators

In this paper, we derive the general formulae for the moments of a linear functional of the Stein-rule and positive-part Stein-rule estimators (i.e., h′ b SR and h′ b PSR) in a different way from Phillips (1984). The MSE dominance of h′ b PSR over h′ b SR and a sufficient condition for h′ b SR to dominate a linear functional of the OLS estimator ( h′ b) are shown. Using the formulae for the moments, we evaluate numerically the first four moments of b SR and b PSR, and examine the bias, variance, MSE, skewness, and kurtosis.

On an adjustment of degrees of freedom in the minimim mean squared error ertimator

In this paper, we consider an adjustment of degrees of freedom in the minimum mean squared error (MMSE) estimator, We derive the exact MSE of the adjusted MMSE (AMMSE) estimator, and compare the MSE of the AMMSE estimator with those of the Stein-(SR), positive-part Stein-rule (PSR) and MMSE estimators by numerical evaluations. It is shown that the adjustment of degrees of freedom is effective when the noncentrality parameter is close to zero, and the MSE performance of the MMSE estimator can be improved in the wide region of the noncentrality parameter by the adjustment, ft is also shown that the AMMSE estimator can have the smaller MSE than the PSR estimator in the wide region of the noncentrality parameter

A Comparison of the Stein-Rule and Positive-Part Stein-Rule Estimators in a Misspecified Linear Regression Model

In this paper, we examine the performance of the predictive risk of the Steinrule (SR) and positive-part Stein-rule (PSR) estimators when relevant regressors are omitted in the specified model. The exact formula of the predictive risk of the PSR estimator is derived, and the sufficient condition for the PSR estimator to dominate the SR estimator under a specification error is given. It is shown by numerical computation that the PSR estimator seems to be the best choice among the OLS, SR, and PSR estimators even when there are omitted variables.

The positive-part Stein-rule estimator and tests of linear hypotheses

When testing linear restrictions in the multiple regression model the usualF-test, based on the maximum-likelihood estimator, is uniformly most powerful invariant. We show that the counterpart to this test, based on the positive-part Stein-rule estimator, has this same optimal property.