Surveying Professional Forecasters

Abstract

This case illustrates how averaging point forecasts harnesses the wisdom of crowds. Students access data from the Survey of Professional Forecasters (SPF) and compare the performance of the crowd (i.e., the average point forecasts) to the average performance of the individual panelists and the best performer from the previous period.The case is intended for use in a class on forecasting, and the instructor can present it in three ways: with all necessary SPF data cleaned and preprocessed in a student spreadsheet (UVA-QA-0805X, provided with the case); with code (also provided in the student spreadsheet) written by the case authors in R, the statistical computing package, as well as a supplementary handout (UVA-QA-0805H, also provided with the case), which walks students through R code, explaining how to clean and analyze the SPF data; or as a team project to be worked on over several days, providing neither the spreadsheet nor the supplement. Excerpt UVA-QA-0805 Rev. Apr. 7, 2014 SURVEYING PROFESSIONAL FORECASTERS Since 1981, the Wall Street Journal surveyed economists for point forecasts of economic indicators such as gross domestic product (GDP), inflation, and unemployment. These forecasts were scored and ranked annually based on their accuracy. The top-ranked forecasters were celebrated in a special announcement of the complete ranking on the Journal's website, typically followed by a press release by the top forecasters' employers. In 2012, the Journal decided for the first time to average all the forecasts for each indicator and present that set of forecasts as an additional panelist. The economists' average forecast, or “the crowd,” was then ranked alongside the actual panelists. How had the economists' average forecast performed? How did the crowd compare to “chasing the expert”? Among the 49 panelists, it ranked 12th in 2012. Yet Don Leavens and Tim Gill, the top-ranked team in 2011, came in 5th in 2012 (see Exhibits 1 and 2). What did this say about the wisdom of the crowd? . . .