Is item response theory superior to classical test theory in scoring A‐IADL?

Abstract

AbstractBackgroundThe Amsterdam IADL Questionnaire (A‐IADL‐Q) is increasingly being used in Alzheimer disease (AD) trials to assess activities of daily living. The authors of this scale (Sikkes et al., 2012) originally proposed an Item Response Theory (IRT) based scoring. While such an approach has some potential advantages, it requires a complex model, assumptions about population score distributions, and can be difficult to interpret clinically. An alternative to IRT scoring is the Classical Test Theory (CTT) method, which is relatively simple and easier to interpret. The present study compared IRT and CTT for scoring A‐IADL as a function of global cognition in healthy older adults and patients with mild‐to‐moderate AD.MethodAggregated data from three multinational clinical trials including subjects at the preclinical, early symptomatic, and moderate dementia phase of AD were analyzed. A‐IADL and CDR assessments at initial visits were evaluated and participants were classified into three groups based on CDR Global Scores: 0, 0.5, and 1. For IRT scoring, Graded Response Model (GRM) was used to estimate individual’s latent score. For CTT method, scaled average of scored responses were calculated. ROC analysis was conducted to evaluate the utility of each method in distinguishing between CDR groups.ResultThere were a total of 2,694 A‐IADL assessments across the three CDR groups. There was a very high correlation between CTT and IRT estimates of A‐IADL total score (r = 0.996, p < 0.05). As CDR Global Score increased, A‐IADL scores declined for both methods. In IRT scoring, the Item Characteristics Curve showed an overlap for most item responses, and the test information curve was narrow with a peak to the right of theta 0. Importantly, the ROC curve showed slightly better performance for CTT [AUC: 0.829 (0.812‐0.847)] compared to IRT [AUC: 0.809 (0.788‐0.830)].ConclusionThe current study found that CTT scoring of A‐IADL performed slightly better in distinguishing AD populations as a function of global cognition/function. Given the procedural complexities and required assumptions for IRT scoring, the CTT method, which has the advantages of being more straightforward and familiar to clinicians, represents a better choice for most applications of this instrument.