Minimizing Revisions for a Monthly Economic Indicator

Abstract

The Advance Monthly Retail Trade and Food Services Survey (MARTS) publishes estimates of total sales in selected retail and food service industries approximately two working weeks after the reference month. One month later, the MARTS sales estimate is superseded by the preliminary estimate from the Monthly Retail Trade and Food Services Survey (MRTS); the MRTS estimate is further revised to incorporate data from late reporters and to incorporate additional data corrections. Thus, the MARTS estimates serve as a forecast of the MRTS estimates. It is also an economic indicator that is used as input into Gross Domestic Product (GDP). Revisions to corresponding estimates are therefore expected and unavoidable due to the sample design (MARTS is a probability subsample of MRTS) and the compressed data collection timeline. However, revisions that reverse the direction of the month-to-month change are problematic. Consequently, the US Census Bureau is investigating methodological enhancements to the current procedures in an effort to minimize these revisions, specifically focusing on the currently used estimation and imputation procedures. We begin by providing background for both MARTS and MRTS surveys. This includes a brief overview of the applicable sampling designs, imputation methods, and estimation methodologies. We also describe the data collection and review procedures, detailing specialized procedures developed by survey analysts for missing data treatments for selected MARTS nonrespondents. This background information motivates the research. Our research focused on three different empirical evaluations of the MARTS survey methodology. We begin with an evaluation of the currently used synthetic estimator with several alternatives, including more traditional weighting adjustment estimators. This is followed with a proposal for an objective method for identifying high-priority units for analyst imputation. Finally, we present and evaluate two different automatic imputation procedures, designed to replace the subjective analyst imputation procedures: Bayesian hierarchical regression models and ARIMA time series models. To compare the performance of the candidate imputation methods in a pseudo-production setting, we conducted a simulation study using MRTS data. We conclude with some general recommendations and areas for future research. Although our recommendations are specific to MARTS, the research and evaluation methods used, and the parallel testing approaches are not. The empirical evaluations described provide a template for other programs that collect specialized data that exhibit seasonal patterns, without risking confounding due to incorrect simulation model specifications.