Triage-weighted kappa: a more appropriate triage reliability measure

Abstract

Dear Editor, We would like to thank Drs. de Vet and Terwee for giving us the opportunity to clarify several points in regard to our manuscript, in which we systematically evaluated the relationship between the minimal important difference (MID) and the minimal detectable change (MDC), using four prospectively collected longitudinal data sets [1]. Previously, several authors had suggested that the distributional methods can yield an MDC that can accurately approximate the MID determined by the anchor-based approach [2]. Our findings did not support that approach, and thus, like de Vet and Terwee, we recommend using an external anchor of changewhen determining the MID. We further agree that researchers should be encouraged to establish the MID, that this should dictate the interpretation of outcome measures, that the MID and the MDC represent two different concepts (but, nevertheless, may still, at times, yield similar values), and that ‘‘both the MDC and the MID are important benchmarks on the scale of the measurement instrument.’’ Finally, we agree that the MDC cannot replace the MID, the primary conclusion of our study. Nevertheless, there are situations in which the MID is not established, and there is a need to make results interpretable (e.g., determining the number of patients who responded to therapy rather than the difference in the mean change, the significance of which may be elusive). This may arise, for instance, in the context of systematic reviews and metaanalyses. Drs. de Vet and Terwee imply that studies should not be considered until the MID has been established by anchor-based methods. We feel that researchers should be given as many tools as possible to maximize the information value of their research even when the situation is not ideal. It is, therefore, worthwhile to have guidance on the best choice for a distribution-based estimate until an optimal anchorbased MID is established. Regarding our choice of a larger value on the anchor for the MID of clinimetric index than psychometric index, de Vet and Terwee state that ‘‘it is illogical to decide that a ‘small change’ is minimally important when using a psychometric instrument, and a ‘moderate change’ is minimally important when using a clinimetric instrument.’’ However, the score of a clinimetric index is very different from that of a psychometric index, and it is the instrument final score, rather than the scoring of the anchor itself, that counts for the MID. A small change on the anchor may yield a misleading choice of the MID for a clinimetric scale. We showed on the Pediatric Ulcerative Colitis Activity Index, that a change of only one gradation on a single highly weighted item (i.e., blood in stool) may achieve a small change in the global rating of change. This, has no face validity since a change of one gradation of one item cannot possibly be a meaningful change [3,4]. This is not usually the case in psychometric indexes in which items typically receive equal weight, and a change in several items is required to reach a small MID. Drs. de Vet and Terwee apparently assume that there is one ‘‘true’’ cutoff on the global rating of change to establish the MID. There are currently no data to support the choice of either small or moderate change for the MID. Therefore, different values may and should be used when the concept under study is different (such as those in the cases of clinimetric and psychometric indices). Finally, it may have been preferable if ‘‘minimal important change’’ had been chosen as the original term for what we all now call the ‘‘minimal important difference.’’ Nevertheless, the field of measurement includes sufficient terminological confusion without changing a semantic designation that is minimally, if at all, problematic.