Further Evaluation of the Validity and Reliability of the Performance Diagnostic Checklist-Human Services

Abstract

ABSTRACT The Performance Diagnostic Checklist-Human Services (PDC-HS) is primarily an informant-based assessment tool designed to help consultants identify variables contributing to problematic employee performance and formatted specifically for use in human service settings. In an evaluation of the tool’s validity and reliability, a previous study used a series of video vignettes to simulate a consultant using the PDC-HS to troubleshoot performance problems; participants scored each video twice, providing measures of validity, test–retest reliability, and interrater reliability. Although the results suggested that the PDC-HS was valid and reliable, the vignettes may have lacked external validity. In the current study, we replicated and extended previous research to include video-based scenarios from actual consultant interviews using the PDC-HS. Twenty-one participants scored three videos, each with a different set and number of domains indicated for intervention. Approximately one month later, participants scored the same videos to assess test–retest reliability. Results suggest that participants were largely able to identify the problematic PDC-HS domains and that the tool was generally reliable, although scores were somewhat lower than those reported by in previous research. Results also suggest that the tool is most effectively implemented by a consultant with at least some background in behavior analysis.