Detecting the effects of physician training in self-care interviewing skills: Coding of standardized patient (SP) visit recordings versus SP post-visit ratings

Abstract

ObjectiveTo compare how coder ratings of standardized patient (SP) visit recordings and SP ratings of the visits detect primary care physician (PCP) training in self-efficacy enhancing interviewing techniques (SEE IT). MethodsAnalyses of data from 50 PCPs who participated in a randomized controlled trial of SEE IT training, which led to increased SEE IT use during three SP visits 1–3 months post-intervention. Untrained SPs rated SEE IT use post-visit. Subsequently, three trained coders generated a consensus SEE IT rating from visit audio recordings. SPs and coders were blinded to provider study arm, and coders to SP ratings. ResultsSP and coder ratings were correlated (r=0.62). In detecting the intervention effect, the areas under the receiver operating characteristic curve were 0.80 (95% CI 0.74–0.87) and 0.76 (95% CI 0.69–0.84) for consensus coder and SP ratings, respectively (difference 0.04, 95% CI −0.04–0.11; z=1.04, p=0.30). ConclusionSP ratings were not significantly different from coder ratings of SP visit recordings in detecting PCP SEE IT training. Practice implicationsIf similar findings are observed in larger studies, it would suggest a greater role for SP ratings in detecting provider interviewing skills training, given the relative simplicity, low cost, and non-intrusiveness of the approach.