Predicting Multivariate Response in Linear Regression Model

Abstract

Predicting a multivariate response vector in a linear multivariate regression model requires the estimate of the matrix of regression parameters. Stein (Stein, C. (1973). Estimation of the mean of a multivariate normal distribution. Proc. Prague Symp. Asymp. Statist. 345–381), van der Merwe and Zidek (van der Merwe, A., Zidek, J.V. (1980). Multivariate regression analysis and canonical variates. Canadian Journal of Statistics 8:27–39), Bilodeau and Kariya (Bilodeau, M., Kariya, T. (1989). Minimax estimators in the normal MANOVA model. Journal of Multivariate Analysis 28:260–270) and Konno (Konno, Y. (1990). On estimation of a matrix of mean. Unpublished manuscript; Konno, Y. (1991). On estimation of a matrix of normal means with unknown covariance matrix. J. Multi. Analysis 36:44–55) have shown that their shrinkage estimators perform better than the least squares estimator. Recently, Breiman and Friedman (Breiman, L., Friedman, J. H. (1997). Predicting multivariate responses in multiple regression. J. Roy. Statist. Soc. Ser. B 59:3–54) proposed another class of shrinkage estimators, called C&W-GCV estimators. Through extensive simulations, they have showed that their C&W-GCV estimator performs better than the FICYREG estimator of van der Merwe and Zidek (van der Merwe, A., Zidek, J. V. (1980). Multivariate regression analysis and canonical variates. Canadian Journal of Statistics 8:27–39), the reduced rank regression method of Anderson (Anderson, T. W. (1951). Estimating linear restrictions on regression coefficients for multivariate normal distribution. Ann. Math. Statist., 22:327–351 (Correction in Ann. Statist. (1980), 8, 1400). Estimating linear restrictions on regression coefficients for multivariate normal distribution. Ann. Math. Statist. 22:327–351. (Correction in Ann. Statist. (1980), 8, 1400)), the component-wise ridge regression and the partial least squares. They, however, did not include in their comparisons, the minimax estimators of Bilodeau and Kariya (Bilodeau, M., Kariya, T. (1989). Minimax estimators in the normal MANOVA model. Journal of Multivariate Analysis 28:260–270) and Konno (Konno, Y. (1990). On estimation of a matrix of mean. Unpublished manuscript; Konno, Y. (1991). On estimation of a matrix of normal means with unknown covariance matrix. J. Multi. Analysis 36:44–55). In this article, we compare C&W-GCV estimator with two invariant minimax estimators and show that C&W-GCV does not perform as well as the two minimax estimators unless the number of response variables is fairly small compared to the number of independent variables and the sample size is small.