Performance Assessment of Pharmacotherapy: Results from a Content Validity Survey of the Psychopharmacotherapy-Structured Clinical Observation (P-SCO) Tool.

Abstract

The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) tool is designed to assess performance of a medication management visit and to enhance feedback. Prior research indicated that the P-SCO was feasible to implement in a resident clinic and generated behaviorally specific, high-quality feedback. This research also highlighted problems with some of the instrument's items. This study seeks to improve the items. The authors initially revised the P-SCO items based on the problems identified by a prior study. Next, these items were iteratively modified by experts in clinical pharmacotherapy and educational assessment. Forty-five items emerged. Finally, faculty attending an annual department education retreat rated each item on its relevance (4-point scale) and provided comments on how the item might be revised. For final inclusion, an item must have met a quantitative threshold (i.e., content validity index equal to or greater than 0.8 and the lower end of the asymmetric confidence interval equal to or greater than 3.0) and received comments that were supportive. Forty-one of the 45 items had strong quantitative support. However, the comments endorsed lumping a number of items in order to decrease overlap between items and to shorten the instrument. This process resulted in the further elimination of 15 items. The revised 26-item P-SCO builds upon prior evidence of feasibility and utility and now possesses additional evidence of content validity. The use of the tool should enhance feedback and improve the capacity of educational programs to assess performance.