Asymmetrical reliability of the Alda score favours a dichotomous representation of lithium responsiveness.

Abraham Nunes,Martin Alda,Thomas Trappenberg,Vincenzo De Luca

doi:10.1371/journal.pone.0225353

Abstract

The Alda score is commonly used to quantify lithium responsiveness in bipolar disorder. Most often, this score is dichotomized into "responder" and "non-responder" categories, respectively. This practice is often criticized as inappropriate, since continuous variables are thought to invariably be "more informative" than their dichotomizations. We therefore investigated the degree of informativeness across raw and dichotomized versions of the Alda score, using data from a published study of the scale's inter-rater reliability (n = 59 raters of 12 standardized vignettes each). After learning a generative model for the relationship between observed and ground truth scores (the latter defined by a consensus rating of the 12 vignettes), we show that the dichotomized scale is more robust to inter-rater disagreement than the raw 0-10 scale. Further theoretical analysis shows that when a measure's reliability is stronger at one extreme of the continuum-a scenario which has received little-to-no statistical attention, but which likely occurs for the Alda score ≥ 7-dichotomization of a continuous variable may be more informative concerning its ground truth value, particularly in the presence of noise. Our study suggests that research employing the Alda score of lithium responsiveness should continue using the dichotomous definition, particularly when data are sampled across multiple raters.

Highlights

The Alda score is a validated index of lithium responsiveness commonly used in bipolar disorder (BD) research [1]
Beyond an observation noise of approximately α > 3.52, one can see that the dichotomized lithium response definition retains greater mutual information (MI) between the true and observed labels, compared to the raw representation
Under agreement levels that are constant across the (x, y) domains, one can observe that MI of dichotomized representations of the variables are generally lower than their continuous counterparts

Summary

Introduction

The Alda score is a validated index of lithium responsiveness commonly used in bipolar disorder (BD) research [1]. The second component is the “B” subscale that attempts to qualify the degree to which any improvement was causally related to lithium. The total Alda score is computed based on these two subscale scores, and takes integer values between 0 and 10. Many studies that employ the Alda score as a target variable dichotomize it, such that individuals with scores 7 are classified as “responders,” and those with scores < 7 are “non-responders.”

Methods

Results

Discussion

Conclusion