On MAPE-R as a measure of cross-sectional estimation and forecast accuracy

Abstract

Journal of Economic and Social Measurement 32 (2007) 219–233 IOS Press On MAPE-R as a measure of cross-sectional estimation and forecast accuracy 1 Charles D. Coleman a and David A. Swanson b a Population Division, US Census Bureau, CENHQ Rm. 6H480C, Washington, DC 20233, USA E-mail: charles.d.coleman@census.gov b Department of Sociology and Edward J. Blakely Center for Sustainable Suburban Development, University of California Riverside, Riverside, CA 92521, USA E-mail: David.swanson@ucr.edu Both demographers and economists evaluate the accuracy of their respective forecasts with measures like mean square error, root mean square error, mean absolute percent error, and mean algebraic percent er- ror. However, demographers tend to approach the issue of forecasting very differently than do economists. Two of the distinctive features of the demographic tradition are the use of the cohort-component method (instead of time-series models) and an emphasis on cross-sectional forecasts (instead of forecasts aggre- gated over time). From the perspective of this demographic tradition, we examine “MAPE-R” (Mean Absolute Percent Error-Rescaled), a recently developed measure of accuracy designed to overcome short- comings noted in “MAPE” (Mean Absolute Percent Error), a measure commonly used to evaluate the accuracy of population estimates and forecasts. We show that MAPE-R can be calculated simply, thus overcoming the cumbersome calculation procedure used in its introduction and noted as a feature needing correction. We find this closed form expression for MAPE-R to be a member of the family of power mean-based accuracy measures. This enables it to be placed in relation to other members of this family, which includes HMAPE (Harmonic Mean Absolute Percent Error), GMAPE (Geometric Mean Absolute Percent Error), and MAPE. Given that MAPE-R was designed to be robust in the face of outliers, it is not surprising to find that it is a valid estimator of the median of the distribution(s) generating the absolute percent errors. Simulation studies suggest that MAPE-R is a far more efficient estimator of this median than MEDAPE (Median Absolute Percent Error). Because the Box-Cox transformation on which MAPE- R depends is known to be unstable, we suggest that this represents a line of further research into GMAPE, which, like MAPE-R, is subject neither to the shortcomings observed for MAPE nor to the instability of the Box-Cox transformation. While further lines of research are called for, nothing in our examination of MAPE-R here rules out its use. It also meets the National Research Council’s major criteria as a summary measure of accuracy. It is subject to some cautions, but these are no more restrictive than those affecting other accuracy measures, many of which are widely used and have been for some years. 1 The views expressed in this paper are those of the authors and not necessarily those of the US Census Bureau or the University of California Riverside. An earlier version of this paper was presented at the 2004 meeting of the Southern Demographic Association, Hilton Head, South Carolina., October 14th–16th. We are grateful for comments provided by Tom Bryan, Jeff Tayman, Charlie Barr, Zhenlin Yang, and Tucker McElroy as well as comments and suggestions provided by the editorial staff of the Journal of Economic and Social Measurement and anonymous reviewers. 0747-9662/07/$17.00  2007 – IOS Press and the authors. All rights reserved