Evaluating Agreement Between Medical Record Diagnoses and Independent Evaluator Diagnoses in a Community-Based Effectiveness Study

Abstract

ABSTRACT Background Youth anxiety disorders are prevalent but undertreated. Poor diagnostic agreement between clinicians and researchers may contribute to misdiagnosis and mistreatment. However, few studies of diagnostic agreement focus specifically on anxiety disorders. Objective The current study evaluates diagnostic agreement between community clinicians and independent evaluators (IE) associated with a research study. Method IEs administered the Anxiety Disorders Interview Schedule for DSM-5, Child Version, Child and Parent Forms to youth participating in a randomized controlled trial. Diagnostic agreement was assessed using Cohen’s kappas. Youth (N = 103) were about 14.3 years old (SD = 1.6), largely cis-gender female (66%, n = 68), heterosexual (57.3%, n = 59), Hispanic/Latinx (53.4%, n = 55), and White (53.4%, n = 55). Results IEs assigned significantly more diagnoses than community clinicians (t(99)=-12.738, d = 1.66, p < .001). Individual diagnoses were collapsed into diagnostic clusters: anxiety, depression, trauma and stressor, obsessive-compulsive, and behavioral disorders. There was no agreement for anxiety, trauma and stressor, obsessive-compulsive clusters (ks < 0.20). The depression cluster showed minimal agreement (k = 0.28, p < .001), and behavioral disorders showed weak agreement (k = 0.49, p < .001). There was no agreement for four individual anxiety disorder diagnoses (ks < 0.20) except for panic disorder, which had minimal agreement (k = .32, p < .001). Four additional anxiety disorder diagnoses could not be analyzed due to no diagnoses in one or both settings. Discussion This study shows diagnostic agreement is poor between clinicians and researchers, especially for anxiety disorders. Findings suggest that semi-structured interviews can detect diagnoses that are being missed by clinicians. Future work should seek to identify and address barriers to implementation of evidenced-based assessments in community settings.