Sociodemographic Differences in Respondent Preferences for Survey Formats: Sampling Bias and Potential Threats to External Validity

Abstract

ObjectiveTo explore sampling bias as a result of survey format selection by examining associations between characteristics of people aging with long-term physical disability (PAwLTPD) and their preferences for phone or web-based survey format. DesignA cross-sectional study using a secondary data analysis approach. SettingData were from an ongoing longitudinal cohort study conducted in the community. ParticipantsConvenience sampling was used. PAwLTPD who participated in year 2 of the longitudinal cohort study were included. Inclusion criteria were age 45-65 years, English speaking, and self-reported onset of a physical disability at least 5 years prior to study recruitment. Two participants completed the survey using both phone and web formats and were thus excluded; 387 participants (N=387) were included in the analysis. InterventionsNot applicable. Main Outcome MeasuresChoice of survey format and demographics (age, sex, race and ethnicity, marital status, living arrangement, socioeconomic status) were collected in addition to self-rated physical health. ResultsParticipants were on average 58.2±5.6 years old. A total of 33% were male, and 62% were White. Approximately 40% of participants completed phone surveys. The phone survey group was significantly older (t=−4.76, P<.001) and had lower education (U=11133, z=−6.65, P<.001) and lower self-rated physical health (U=15420, z=−2.38, P=.017) than the web survey group. Participants who were White (χ2=60.69; df=1; P<.001; odds ratio [OR], 0.18) or were in a long-term relationship were less likely to choose phone surveys (χ2=42.20; df=1; P<.001; OR, 0.21). Those who earned $10,008 or less annually (χ2=53.90; df=1; P<.001; OR, 5.22) or who lived alone (χ2=36.26; df=1; P<.001; OR, 3.64) were more likely to choose phone surveys. Participants with paid work (χ2=16.81, df=1, P<.001) tended to select web-based surveys, while those on disability leave (χ2=9.61, df=1, P<.01) were more likely to choose phone surveys. ConclusionsSociodemographics are associated with survey format choice in PAwLTPD. Findings largely support the existing understanding of digital literacy but also provide insight into the potential occurrence of sampling bias when multiple survey format options are not offered. These findings have implications for investigators who aim to reach a more representative sample of people with disabilities.