Portfolio Performance Assessment: Statistical Issues and Methods for Improvement

Abstract

Listen

Translate article

A central problem for performance-oriented portfolio formation modeling is to backtest the performance of a forecast model/method over a large sample of securities for a relatively long time period. In a multifactor, multistyle, multirisk modeling framework, backtest assessments of return forecast performance potential are complicated by measurement error, pervasive multicollinearity, specification error, and even omitted variable error. This chapter shows how the matched control methodology commonly used in controlled and partially controlled studies can be adapted to isolate well the realized return response to a return forecast with complete suppression of covariation distortion. This chapter formulates and then illustrates the use of a power optimizing mathematical program that transforms a cross section of forecast rank-ordered fractile portfolios into an associated relative rank-ordered cross section of control-matched portfolios. In contrast to the linearity/bilinearity assumption in most multivariate stock return models and the strong distribution assumptions in regression estimation, the matched control methodology requires no distribution or functional form assumptions. Estimation benefits relative to conventional multivariate regression assessments include (1) complete elimination of collinearity distortion, (2) mitigation of measurement error, (3) and mitigation of specification errors for known variable dependencies of unknown functional form. Particularly important in the test example illustrated here is the ability to remove variation in the dividend–gain mix and thereby control for systematic tax effects without estimating the marginal tax rate for either dividends or gains. The benefits of the matched control methodology are illustrated for an eight-variable return forecast model that provides apparent performance benefits but whose performance assessment is complicated by measurement error, by extreme multicollinearity, and by possible specification and completeness issues including a possible yield tilt (tax tilt) from the high correlation of the forecast model variables with dividend yield. For the illustrative forecast, the cross sections of realized risky return, realized standard deviations, and skewness coefficients are nonlinear. Moreover, the risky return cross-sections change significantly as different combinations of control variables are imposed on the uncontrolled rank-ordered cross sections, e.g., the slope of the regression of realized returns on forecast score changes from 8.xx with no controls to 16zz with just the three tax controls. In addition to mitigating estimation bias, the primary statistical benefit is greatly increased statistical confidence and power relative to using multivariate regression to try to isolate forecast performance potential.