SME Committee vs. Field Job Analysis Ratings: Convergence, Cautions, and a Call

Abstract

This investigation examined the agreement between job analysis ratings completed by a small committee of subject matter experts (SMEs) and ratings made by a larger sample of field respondents. The results indicated there were very high levels of agreement across six commonly used job analysis rating indices. These findings converge with those in prior studies suggesting that for purposes of obtaining task and KSA ratings, a small committee of SMEs may provide empirical results that agree with those obtained from a larger sample of field respondents. However, cautions are presented for applying a “committee only” strategy in job analysis. Given convergence in conclusions across studies such as the current one thus far, a call for future research which addresses more specific issues is presented.