Value of Expertise for Forecasting Decisions in Conflicts

Kesten C Green,J Scott Armstrong

doi:10.2139/ssrn.675381

Abstract

In important conflicts, people typically rely on experts' judgments to predict the decisions that adversaries will make. We compared the accuracy of 106 expert and 169 novice forecasts for eight real conflicts. The forecasts of experts using unaided judgment were little better than those of novices, and neither were much better than simply guessing. The forecasts of experts with more experience were no more accurate than those with less. Speculating that consideration of the relative frequency of decisions might improve accuracy, we obtained 89 forecasts from novices instructed to assume there were 100 similar situations and to ascribe frequencies to decisions. Their forecasts were no more accurate than 96 forecasts from novices asked to pick the most likely decision. We conclude that expert judgment should not be used for predicting decisions that people will make in conflicts. Their use might lead decision makers to overlook other, more useful, approaches.

Highlights

While it is attractive to think that if we can find the right expert we can know what will happen, Armstrong (1980) in a review of evidence from diverse subject areas was unable to find evidence that expertise, beyond a modest level, improves experts’ forecasting accuracy
Our talks were to academics and students at Lancaster University (19 usable responses), Manchester Business School (18), Melbourne Business School (6), Royal New Zealand Police College educators (4), Harvard Business School alumni (8), conflict management practitioners in New Zealand (7), and attendees at the International Conference on Organizational Foresight in Glasgow (15)
We excluded all responses from those who expected accuracy to be less than 28% for any method as it seems implausible that the forecasts of any method would be worse than chance

Summary

Introduction

While it is attractive to think that if we can find the right expert we can know what will happen, Armstrong (1980) in a review of evidence from diverse subject areas was unable to find evidence that expertise, beyond a modest level, improves experts’ forecasting accuracy. Asking an expert to predict what will happen in a conflict seems a reasonable thing to do. Evidence from surveys suggests that forecasts of decisions in conflicts are typically based on experts’ unaided judgments (Armstrong, Brodie, and McIntyre 1987).

Results

Conclusion