Non-Standard Diagnostic Assessment reliability in psychiatry: A study in a Brazilian outpatient setting using Kappa

Abstract

IntroductionThe use of Structured Diagnostic Assessments (SDAs) is a solution for unreliability in psychiatry and the gold standard for diagnosis. However, except for studies between the 50s and 70s, reliability without the use of Non-SDAs (NSDA) is seldom tested, especially in non-Western, Educated, Industrialized, Rich, and Democratic (WEIRD) countries.ObjectivesWe aim to measure reliability between examiners with NSDAs for psychiatric disorders.MethodsWe compared diagnostic agreement after clinician change, in an outpatient academic setting. We used inter-rater Kappa measuring 8 diagnostic groups: Depression (DD: F32, F33), Anxiety Related Disorders (ARD: F40–F49, F50–F59), Personality Disorders (PD: F60–F69), Bipolar Disorder (BD: F30, F31, F34.0, F38.1), Organic Mental Disorders (Org: F00–F09), Neurodevelopment Disorders (ND: F70–F99) and Schizophrenia Spectrum Disorders (SE: F20–F29) (Check table 1 about diagnosis hyerarchy and observed frequency in sample). Cohen’s Kappa measured agreement between groups, and Baphkar’s test assessed if any diagnostic group have a higher tendency to change after a new diagnostic assessment. This research was approved by IPUB’s ethical committee, registered under the CAAE33603220.1.0000.5263, and the UTN-U1111-1260-1212.ResultsWe analyzed 739 reevaluation pairs, from 99 subjects who attended IPUB’s outpatient clinic. Overall inter-rater Kappa was moderate, and none of the groups had a different tendency to change (Check table 2 for diagnostic change distribution). Our tests achieved the followinf results: Cohen Kappa 0.70, IC: 0.66– 0.74; Weighted Kappa 0.72, IC:0.72 – 0.72; Bhapkar Test X² = 5.98, Df = 7, P-value = .55; Achieved Power (w=0.1): 0.93 Table 2Agreement between examiners for eight diagnostic groupsARDBDDDDRDNDOrganicPDSSDARD393902033BD11547322410DD9107100259DRD02040002ND121051116Organic010032005PD421100333SSD520112854192ConclusionsNSDA evaluation was moderately reliable, but the lack of some prevalent hypothesis inside the pairs raised concerns about NSDA sensitivity to some diagnoses. Diagnostic momentum bias (that is, a tendency to keep the last diagnosis observed) may have inflated the observed agreement.Disclosure of InterestNone Declared