The effects of performance sampling methods on frequency estimation, probability estimation, and evaluation of performance information

Abstract

This paper presents the results of two experiments investigating the effects of raters' methods of performance sampling on their ability to make accurate estimates and evaluations concerning the observed performance of multiple ratees. In two experiments, 120 college undergraduates took part in a performance observation task where they sampled the sales performances of six ratees (three good and three poor performers). The number of performance instances sampled for each ratee and the level of performance exhibited by the ratee (good, poor) were experimentally manipulated, in order to investigate their impact on raters' ability to estimate (a) the frequency of poor performance instances; (b) the frequency of the total sample (good and poor) of performance instances; and (c) the probability of poor performance (poor/total). Results indicated that estimates which were based upon a greater amount of observed information were more accurate than those based upon less information, whether probability, frequency, or sample size information was being estimated. In addition, subjects who equally sampled all ratees made more accurate relative estimates of ratee performance than did those who unequally sampled good and poor performing ratees. Finally, subjects were found to use performance probability information to produce accurate evaluative ratings concerning the relative levels of ratee performance, although performance ratings were found to be less reliable measures of relative ratee performance than were probability, frequency, and sample size estimates. Implications regarding performance sampling and the use of performance appraisal formats that elicit estimates concerning the frequency and probability of ratee performance are discussed.