Why Are Estimates Always Wrong: Estimation Bias and Strategic Misestimation

Abstract

Many people view an estimate as a quick guess that no one believes anyhow.. But producing a viable estimate is core to project success as well as ROI determination and other decision making. In decades of studying the art and science of estimating it has become obvious that: 1) Most people don’t like to estimate; Most people don’t know how to estimate, 3) Those that estimate are often always wildly optimistic, full of unintentional bias; 5)strategic misestimating provides misleading estimates when it occurs and 5) Viable estimates can make projects successful, outsourcing more cost effective, and help businesses make the most informed decisions. That is why metrics and models are essential to organizations, providing the tempering with that outside view of reality that is recommended by Daniel Kahneman in his Nobel prize winning work in estimation bias and strategic mis-estimation. . I. EXPERTS ARE PROVIDING BIASED ESTIMATES One can merely scan the news and see the daily disasters that come from poor estimates. And as much as they try people continue to produce such poor estimates. It’s not that people are not capable. It is that they are hardwired to produce optimistic estimates. They think that this time things will go better. This time the mistakes of the past will not occur. They think they are smarter this time and they have this one under control. But Alas, history too often repeats itself yielding lost time, lost money, and lost careers. Time and time again we see biased estimates coming from both act experts and technologists. In the article Delusions of Success: How Optimism Undermines Executives' Decisions the authors point out people routinely exaggerate the benefits while discounting the costs. The authors suggest that tempering, that is providing an outside view such as past measurement results, traditional forecasting risk analysis and parametric modelling can help. Additionally they suggest not removing optimism but balancing optimism with realism. Kahneman and Tversyk also found that judgement errors are systematic and predictable, not random and that such errors continue even when the estimators are aware of them. The root cause is that each new venture is viewed as completely unique And that they took an “inside view” focusing on the components being estimated rather than the outcomes of similar completed actions. Applying Estimating ranges: The first and simplest method of helping to mitigate estimation bias is to have estimators estimate not single points but ranges of best case likely case and worst-case. Having to think about those cases, when everything goes right and when everything goes wrong, helps people recognize when they might be biased in the positive sense. Douglas Hubbard, author of the book quote how to measure anything quote suggests that perception that measurement is a point value is a key reason why many things are perceived as immeasurable. That is, if we are willing to estimate a range including our uncertainty we can estimate many things. Figure 1 illustrated the improved decision making from reduced uncertainty. Figure 1 Measurement Reduces Risk, Copyright Hubbbard II. REFERENCE CLASS FORECASTING Reference class forecasting is a technique that shows that the best predictor of performance is actual performance of implemented comparable projects. This “outside view” focuses on the outcomes of completed analogous projects. They suggest that the prior projects should be analogous and that these projects should be used to compute a probability distribution. Then by comparing the new projects to the range of completed projects one can identify probable estimation bias. Parametric estimation model such as SEER provide a superset of reference class forecasting, allowing adjustment for people, process, and product issues hitting the new program. III EXPLAINATIONS FOR POOR ESTIMATING Flybjerg identified three explanations for poor estimating: 1. Technical: Inadequate data & Models (Vanston) 2. Psychological: Planning Fallacy, Optimism Bias causes belief that they are less at risks of negative events 3. Political / Economic: Strategic misrepresentation tendency to underestimate even when experienced with similar tasks overrunning (Flyvberg) Figure 2 Estimation Bias, Not Technical Drive Many Estimates, Copyright Flybjerg In the world of IT estimation adequate models are available, psychological poor estimating is the frequent culprit, and political / economic strategic misrepresentation is a regular occurrence often caused by management dictating the functionality, the resources, and the schedule without regard for the resulting quality or the possibility of even achieving the plan. This violates the “Iron triangle of project management” The iron triangle points out that if you change one dimension: resources, scope or schedule quality suffers. Of course, in software development there is a minimum time complete particular software development. So quality as well as success can be on the line.