Comparison of Precision and Accuracy of Five Methods to Analyse Total Score Data

Gustaf J Wellhagen,Maria C Kjellsson,Mats O Karlsson

doi:10.1208/s12248-020-00546-w

Gustaf J Wellhagen, Maria C Kjellsson + Show 1 more

Open Access

https://doi.org/10.1208/s12248-020-00546-w

Copy DOI

Journal: AAPS PharmSci	Publication Date: Dec 17, 2020
Citations: 2	License type: open-access

Affiliation: Uppsala University

Abstract

Total score (TS) data is generated from composite scales consisting of several questions/items, such as the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). The analysis method that most fully uses the information gathered is item response theory (IRT) models, but these are complex and require item-level data which may not be available. Therefore, the TS is commonly analysed with standard continuous variable (CV) models, which do not respect the bounded nature of data. Bounded integer (BI) models do respect the data nature but are not as extensively researched. Mixed models for repeated measures (MMRM) are an alternative that requires few assumptions and handles dropout without bias. If an IRT model exists, the expected mean and standard deviation of TS can be computed through IRT-informed functions—which allows CV and BI models to estimate parameters on the IRT scale. The fit, performance on external data and parameter precision (when applicable) of CV, BI and MMRM to analyse simulated TS data from the MDS-UPDRS motor subscale are investigated in this work. All models provided accurate predictions and residuals without trends, but the fit of CV and BI models was improved by IRT-informed functions. The IRT-informed BI model had more precise parameter estimates than the IRT-informed CV model. The IRT-informed models also had the best performance on external data, while the MMRM model was worst. In conclusion, (1) IRT-informed functions improve TS analyses and (2) IRT-informed BI models had more precise IRT parameter estimates than IRT-informed CV models.

Highlights

Composite scale data is made up of many questions/ items with categorical responses that can be summed up through an algorithm into a total score (TS), which is discrete and bounded
As the IRTinformed functions transform the estimates of the model to the Item response theory (IRT)-disease progression scale, parameter estimates can be compared between different model types
The I-continuous variable (CV) model was better than the I-bounded integer (BI) model at predicting the size of the drug effect without bias, while their respective precision was similar

Summary

Introduction

Composite scale data is made up of many questions/ items with categorical responses that can be summed up through an algorithm into a total score (TS), which is discrete and bounded. Item response theory (IRT) models make use of itemlevel information; well-constructed IRT models are considered the most informative way of analysing such data. They map the disease severity to one or several latent variable(s) (Ψ). They are complex to develop, may be difficult to estimate due to a large number of parameters, may take long time to run and sometimes the item-level data is unavailable. The modeller can turn to alternative models: continuous variable (CV), bounded integer (BI) [1] or less mechanistic models such as mixed models for repeated measures (MMRM)

Methods

Results

Discussion

Conclusion