Experienced Forecasters Research Articles

Testing the Conditions for Acquiring Intuitive Expertise in Judgment

Evidence from psychological studies of both novice and expert forecasters often show that people are overconfident in their ability to make accurate judgmental predictions about future events. In contrast, evidence from much cognitive engineering research appears to show that experienced performers are reasonably well adapted to their task environments and do not display many of the cognitive biases, such as overconfidence, evident in the psychological literature. We present the results of a study providing evidence for both views, and more importantly, for the conditions under which experienced forecasters will exhibit high levels of calibration (little overconfidence) versus poorer calibration and thus overconfidence. In short, good calibration resulted when forecasters were required to draw solely upon high validity base-rate information gained through many years of repetitive exposures to a prediction task, yet displayed overconfidence when they were also provided case-specific information with high salience and interest but little or no predictive validity.

What do professional forecasters' stock market expectations tell us about herding, information extraction and beauty contests?

Abstract We study how professional forecasters form equity market expectations based on a new micro-level dataset which includes rich cross-sectional information about individual characteristics. We focus on testing whether agents rely on the beliefs of others, i.e., consensus expectations, when forming their own forecast. We find strong evidence that the average of all forecasters' beliefs influences an individual's own forecast. This effect is stronger for young and less experienced forecasters as well as forecasters whose pay depends more on performance relative to a benchmark. Further tests indicate that neither information extraction to incorporate dispersed private information, nor herding for reputational reasons can fully explain these results, leaving Keynes' beauty contest argument as a potential candidate for explaining forecaster behavior.

What do Professional Forecasters' Stock Market Expectations tell us about Herding, Information Extraction and Beauty Contests?

We study how professional forecasters form equity market expectations based on a new micro-level dataset which includes rich cross-sectional information about individual characteristics. We focus on testing whether agents rely on the beliefs of others, i.e., consensus expectations, when forming their own forecast. We find strong evidence that expectations about the average of all forecasters' forecasts influences an individual's own forecast and that this effect is stronger for young and less experienced forecasters as well as forecasters whose pay depends on performance relative to a benchmark. Further tests indicate that neither information extraction to incorporate otherwise dispersed private information, nor herding for reputational reasons can fully explain these results, leaving Keynes' beauty contest argument as a potential candidate for explaining forecaster behavior.

Expertise, Spatial Ability and Intuition in the Use of Complex Visual Displays

In a naturalistic setting, a large group of U.S. Navy forecasters (N=21) performed a weather forecasting task and saved copies of the visual displays that they consulted to complete the task. On the basis of structured interviews with the forecasters, quantitative analysis of the information content of the displays they used, and psychometric measures of their spatial ability, we found two key results. First, forecasters of lower spatial ability populated their forecasting displays with more extraneous meteorological variables than did forecasters of higher spatial ability. Second, we found quantitative support for the previously qualitatively observed trend for more experienced forecasters to exhibit a different temporal pattern of display use. Experienced forecasters tend to consult observations before models, and less experienced forecasters show the reverse trend. This work implies that forecasters of lower spatial ability may benefit from training that teaches them to display only the variables they need to complete a task, and that it may be beneficial to customize forecasting displays to the spatial ability of their users.

The Dynamics of Overconfidence: Evidence from Stock Market Forecasters

As a group, market forecasters are egregiously overconfident. In conformity to the dynamic model of overconfidence of Gervais and Odean (2001), successful forecasters become more overconfident. What's more, more experienced forecasters have learned to be overconfident, and hence are more susceptible to this behavioral flaw than their less experienced peers. It is not just individuals who are affected. Markets also become more overconfident when market returns have been high.

The Contributions of Education and Experience to Forecast Skill

Abstract An analysis of nine semesters of temperature and precipitation forecasts at the State University of New York at Albany has been conducted with the goal of investigating the dependence of forecasting skill on education and experience. The results show that forecast skill is largely determined by experience. The relative advantage of highly experienced forecasters is secured by virtue of the larger set of cases from which they may draw upon: given a set of forecast information (e.g., moisture, winds and cloud cover), such a forecaster is in a better position to maximize linear consistency between that information and the expected evolution of surface temperature and precipitation (given similar conditions, make a similar forecast) than someone with less forecasting experience. However, the experienced forecaster also gains substantially by recognizing those instances in which these linear relationships no longer apply and by forecasting accordingly. Such instances can often be recognized using simp...

METEOR: An Artificial Intelligence System for Convective Storm Forecasting

Abstract An experienced forecaster can use several different types of knowledge in forcing. First, there is his theoretical understanding of meteorology, which is well entrenched in current numerical models. A second type is his “local knowledge,” gained over years of experience, of how weather is likely to form in his forecast area. This kind of local familiarity is not easily captured with traditional numeric techniques, but might provide additional insights for prediction that someone unfamiliar with the area might not have. A third type of knowledge is how to interpret forecast tools already in use. This might include knowledge of the tool's limitations and how it works in a particular locale. Capturing these types of knowledge is important in building computing systems that can serve as intelligent consultants to forecasters. This paper describes a prototype system, called METEOR, that incorporates all these types of knowledge to predict the location, severity, and motion of convective storms in Albe...

Surface pressure anomaly pattern types and their application In forecasting daily rainfall distribution over India

Daily pressure anomaly charts for July and August are objectively classified into ten the large scale types for each month, utilising thirty years (1946-75) of data, through a chart-to-chart correlation method proposed by Lund (1963) .Average spatial distribution of rainfall associated with each of the pressure anomaly types are computed for 31 meteorological sub-divisions covering entire contiguous India using 15 years (1958-72) if data. Usefulness of these rainfall distribution type conditional climatologies in forecasting precipitation distribution over 10 sub-divisions lying in western and central parts of the country for the next 24 hours, is verified using 3 years (1973-75) of charts. The verification reveals that the forecasts as skillful as given by an experienced forecaster using all synoptic information can be made by simple stratification of pressure anomaly charts.