First impressions in software development effort estimation: easy to create and difficult to neutralize

Abstract

Background: Effort estimates of software development work are frequently very inaccurate, which contributes to project management problems and project failures. The most common estimation approach, expert judgment-based estimation, is currently not well understood and more knowledge about how experts create effort estimates may enable better estimation processes and eventually more accurate effort estimates. Aim: Software professionals are affected by their early judgment (first impression) of the effort required for software development work. They are, however, also affected by information presented late (more recently) in the information collection process. This study explores which of these effects, i.e., “first impression” or “recency,” that is likely to be the stronger. This is studied in a context where the information leading to the first impression and the update with more recent information are biased in opposite directions. Method: We completed four experiments. All experiments included the following steps: 1) Creation of a biased first impression through comparisons of the task to be estimated with either much too low or too high reference effort values. 2) Comparisons of the task to be estimated with reference effort values in the opposite direction. Those comparing the task with a much too low reference value in Step 1 now compared the task with a much too high reference value. 3) Estimation of the most likely use of effort. Results: The results suggest that the first impression is stronger than the recency effect, but that there is a noticeable effect from the subsequent (more recent) comparison. We also observed that the explicit instruction to “forget” the information leading to the biased first impression had the paradoxical effect of increasing the strength of the first impression. Conclusions: It is essential for software professionals to ensure that their first impression of a project's development effort is based on comparisons with representative reference values and objects, as first impressions are easy to create but difficult to remove. Software managers should not believe that it will be possible to remove the impact of an incorrect first impression on the effort estimate by instructing the estimator to forget the initial information. The evidence suggests that such instructions would rather lead to a stronger impact from the incorrect first impression.