A novel method for expediting the development of patient-reported outcome measures and an evaluation of its performance via simulation.

Lili Garrard,Marjorie J Bott,Larry R Price,Byron J Gajewski

doi:10.1186/s12874-015-0071-5

Abstract

BackgroundDeveloping valid and reliable patient-reported outcome measures (PROMs) is a critical step in promoting patient-centered health care, a national priority in the U.S. Small populations or rare diseases often pose difficulties in developing PROMs using traditional methods due to small samples.MethodsTo overcome the small sample size challenge while maintaining psychometric soundness, we propose an innovative Ordinal Bayesian Instrument Development (OBID) method that seamlessly integrates expert and participant data in a Bayesian item response theory (IRT) with a probit link model framework. Prior distributions obtained from expert data are imposed on the IRT model parameters and are updated with participants’ data. The efficiency of OBID is evaluated by comparing its performance to classical instrument development performance using actual and simulation data.Results and Discussion The overall performance of OBID (i.e., more reliable parameter estimates, smaller mean squared errors (MSEs) and higher predictive validity) is superior to that of classical approaches when the sample size is small (e.g. less than 100 subjects). Although OBID may exhibit larger bias, it reduces the MSEs by decreasing variances. Results also closely align with recommendations in the current literature that six subject experts will be sufficient for establishing content validity evidence. However, in the presence of highly biased experts, three experts will be adequate.ConclusionsThis study successfully demonstrated that the OBID approach is more efficient than the classical approach when the sample size is small. OBID promises an efficient and reliable method for researchers and clinicians in future PROMs development for small populations or rare diseases.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-015-0071-5) contains supplementary material, which is available to authorized users.

Highlights

Developing valid and reliable patient-reported outcome measures (PROMs) is a critical step in promoting patient-centered health care, a national priority in the U.S Small populations or rare diseases often pose difficulties in developing PROMs using traditional methods due to small samples
This study successfully demonstrated that the Ordinal Bayesian Instrument Development (OBID) approach is more efficient than the classical approach when the sample size is small
Two important assumptions are made by Jiang et al [21] for Bayesian Instrument Development (BID) that apply to the OBID simulation setting

Summary

Introduction

Developing valid and reliable patient-reported outcome measures (PROMs) is a critical step in promoting patient-centered health care, a national priority in the U.S Small populations or rare diseases often pose difficulties in developing PROMs using traditional methods due to small samples. As Bayesian inference becomes more popular, limitations arise with the use of classical (i.e. frequentist) methods when developing instruments or PROMs for small populations (e.g., in cases of rare diseases). Since it is not the intent of the authors to provide a comprehensive review of both classical and Bayesian statistical approaches, we focus our discussions on two co-existing issues with the classical approach to confirmatory factor analysis (CFA) in establishing evidence of construct validity: (a) the requirement of large samples, and (b) modeling ordinal data as continuous

Methods

Results

Discussion

Conclusion