Comparing effects of question set order and location within a survey instrument of two commonly used disability question sets among a U.S. population of adults

Abstract

BackgroundIn nationally representative household surveys conducted in the United States (U.S.), two distinct sets of questions are commonly used to identify persons with disabilities. The six-question sequence (6QS) measures, in a yes/no fashion, limitations in cognition, hearing, independent living, self-care, vision, and walking. The Washington Group Short Set (WG-SS) captures more nuanced yet similar information, although instead of including a measure of independent living asks about communication. To date, research has not estimated, among the same set of survey respondents, variations in disability prevalence using these two sets of questions nor how prevalence estimates vary by question set order and placement of these questions at the beginning or end of a survey. Objective/HypothesisThe objective was to examine adjusted differences in disability prevalence among three measures of disability based on the 6QS and the WG-SS question sets, controlling for differences in question set order and placement within a survey. MethodsWe fielded an Internet survey (N = 13,277) in September 2020 that included these questions, but varied question set order and placement among respondents, using four different versions of the survey. We first tested for bivariate differences by survey design between an “any disability” measure as well as between specific types of limitations using Chi square. Finally, we examined pairwise adjusted differences in prevalence estimates. ResultsThe 6QS provided the most consistent prevalence estimates (26%–28%) (p < .05), regardless of survey design. Estimates varied more widely for the WG-SS measures, ranging from 43 to 60% for WG-SS1 and from 10% to 15% for WG-SS2, among survey versions. ConclusionsQuestion set order and placement was not associated with differences in prevalence for the 6QS but was associated with differences in estimates from the WG-SS. Further research is needed to understand the possible survey priming effects that might influence estimates from the WG-SS.