Compared to what? A joint evaluation method for assessing quality of life

Abstract

This study tests whether a joint evaluation method for assessing quality of life can stabilize ratings by providing contextual information, thereby helping participants calibrate responses on a rating scale. We also use the method to test for scale recalibration between patients and non-patients. In an Internet survey, participants (N = 1,865) rated a target health condition, either diabetes or obesity, on a 100-point rating scale. Participants either rated several other items on the same rating scale first (joint evaluation), or rated the target condition first (single evaluation). We compared target condition ratings for joint versus single evaluation, as well as the rank position of that item among the other items. We also compared ratings and rankings for patients versus non-patients. The method effectively picked up distinct patterns of scale usage, with evidence of scale recalibration for obesity ratings, but not for diabetes ratings. The stabilizing effects of the method were mixed. For both diabetes and obesity, the joint evaluation task helped stabilize the rank position of the target condition, but not the rating. Results do not conclusively support joint evaluation as a method for reducing noise in rating scale usage, but do support its use for detecting scale recalibration between patients and non-patients.