Do supervisors and independent judges agree on evaluations of therapist adherence and competence in the treatment of cocaine dependence?

Abstract

The current study examined the agreement between supervisors' and independent judges' evaluations of therapist adherence and competence in three treatments of cocaine dependence: supportive expressive therapy (SE), cognitive therapy CT), and individual drug counseling (IDC). We used data from The National Institute on Drug Abuse Collaborative Cocaine Treatment Study (n = 295). Trained and experienced supervisors and independent judges rated treatment fidelity. At the therapist level of analysis, the agreement between supervisors' and independent judges' ratings was weak for SE competence, CT adherence, and CT competence. Moderate relations were found for IDC adherence and competence. Supervisors consistently rated adherence and competence more positively than judges in CT and IDC. The potential bias in supervisors' ratings is discussed.