Evaluating Performance in Writing Drug Information Consultations

Abstract

A form for evaluating written drug information consultations was used to measure the performance of pharmacy students in a drug information service clerkship; the reliability of the evaluation process was also assessed. Drug consultations were written by 74 clerks during a four- or five-week rotation. A total of 147 consultations were written: Nine clerks wrote one consultation, 57 clerks wrote two, and eight clerks wrote three. The consultations were evaluated on 11 criteria by 18 different preceptors using an evaluation form. Performance on each criterion was rated on a four-level scale. Clerk improvement over time, reliability of the instrument, and inter-rater reliability for preceptors were analyzed statistically. The overall mean score for all consultations was 84 7%. Some improvement was observed between the first and second consultations completed by clerks, but the difference was not significant. Reliability of the evaluation process was good (coefficient of alpha = 0.666). There as no significant difference among preceptors. The authors concluded that the written drug information consultation evaluation form was a valid and reliable method for measuring individual and program competence.