Assessing the Validity of Job Ratings: An Empirical Study of False Reporting in Task Inventories

Abstract

Despite the widespread use of task inventories in job analysis, little is known about the validity of the obtained task ratings. One approach for examining the validity of such ratings is the use of a “false reporting” index to identify invalid responding. The purpose of this field experiment was to examine the effects of the type of frequency rating scale and method of task inventory administration on the degree of false reporting in task inventory ratings. A total of 177 Correctional Officers from a state correctional system responded to a 68 item task inventory using frequency and importance rating scales. Five of the items in the task inventory were bogus tasks not performed by the target job and formed a false reporting index. In a 2 × 2 design, the type of frequency rating scale (Relative-Time-Spent vs. Actual-Time-Spent) and method of task inventory administration (anonymous vs. identified) were manipulated. Analysis of variance results showed a significantly greater degree of false reporting in Relative-Time-Spent ratings. No significant differences in false reporting were found for method of task inventory administration or scale × method interactions. Overall, 45% of respondents indicated that they performed tasks that were not part of the job, which raises concerns about whether job incumbents are capable of providing accurate and complete task rating data.