Commentary on “A new indicator for the measurement of change with ordinal scores”

Abstract

The paper by Ferreira et al. [1] introduces a new statistic—the Indicator of Positive Change (IpC)—that evaluates, atthe group-level, improvement on an ordinal rating scalefollowing some intervention. As the authors point out, it isnot uncommon to see distributional assumptions frominterval or ratio scales improperly applied to ordinal scaledata. Misapplying these assumption leads to the use ofinappropriate statistical procedures for analysis of ordinalscale data, which often results in inaccurate and unwar-ranted interpretations. The IpC is proposed as a newapproach for calculating a standardized statistic that cap-tures the magnitude of group-level improvement on anordinal scale measure without reliance upon distributionalassumptions, which would avoiding these commonproblems.Unfortunately, despite the authors’ claims, the calcula-tion of the IpC does, in fact, incorporate unwarranted non-ordinal assumptions about the scale being evaluated, spe-cifically the assumption of equal distance between pointsthat does not apply to ordinal scales [2]. Further, compu-tation of the IpC in realistic hypothetical scenarios showsthat it can produce irrational results, which weakens theclaim that the IpC can be interpreted as an accurate orappropriate index of subjects’ improvement in an ordinaloutcome over time.As specified by the authors (Equation 1 in [1]), thecomputation of the IpC explicitly incorporates linear dis-tance among categories. That is, the numerator of theequation used to estimate the IpC is calculated under theassumption that the number of categories across which apatient moves from pre-intervention to post-interventionhas a fixed weight, such that moving two categories isweighted twice as much as moving one category, movingthree categories is weighted three times as much as movingone category, and so on. To illustrate, supposing an ordinalscale in which a patient’s current level of disease activity israted by their physician as normal (coded as 1), mild (2),moderate (3), or severe (4), the IpC would weight patientswhose pre/post-intervention rating changed from ‘severe’to ‘mild’ or ‘moderate’ to ‘normal’ twice as much aspatients whose rating changed from ‘severe’ to ‘moderate’or ‘moderate’ to ‘mild’. Furthermore, patients are weightedthe same regardless of whether they move from ‘severe’ to‘moderate’, ‘moderate’ to ‘mild’, or ‘mild’ to ‘normal’.The underlying assumption made by the IpC—that thedistance between all categories is assumed equal, and thusadditive—is not appropriate for an ordinal scale [2]. Forexample, the true distance between people with subjectiveclinical ratings of ‘severe’ and ‘moderate’ symptoms mightbe larger than those rated as ‘moderate’ and ‘mild’; pos-sibly it is smaller, or maybe the distance is, in fact, thesame. But the strict assumption of equivalent distanceamong points on a scale applies only to interval or ratioscale data [2], and so its incorporation into the IpC clearlyviolates the authors’ stated intention to discard non-ordinalassumptions.The IpC also yields problematic, even nonsensicalresults because the denominator, which standardizes thevariable by calculating the total possible change, differ-entially weights changes over time at the bottom and topends of the scale as a consequence of a ceiling effect.Consider two patients rated on the disease activity scaledescribed above—the first patient with a pre-interventionrating of ‘mild’, and the second patient with a pre-