What is the effect of mode of collection on individual response? Results from a cross-sectional survey of third-level students

Abstract

BackgroundFace-to-face lecture-theatre-based sampling is recognised as the best method of data collection for student-based health behaviour surveys. In practice, however, it is highly labour intensive and undertaken at a high cost. To reduce these costs, many universities use web-based approaches for collection of student-based health behaviour information. However, these approaches have continually produced extremely low response rates. The aim of this study was to compare and contrast sociodemographic and risk-taking behaviour information across lecture-theatre-based and web-based data collection methods for the first time. MethodsDegree programmes at one Irish university were randomly selected for inclusion in the study. Following a sample size calculation, students on all undergraduate degree programmes in all years were eligible for inclusion in the study. Students were sampled by probability proportional to size sampling. In March, 2012, 2275 self-completed questionnaires were obtained over 2 weeks by distribution at lecture theatres. A response rate of 51% was achieved. An email with a link to the questionnaire was sent to all students registered to the same university in March, 2012. 333 undergraduate students responded to the web-based questionnaire, yielding a response rate of 2·4% for the undergraduate population. Mantel-Haenszel tests, independent samples t tests, and multivariate logistic regression were used to investigate the effect of answering questions through lecture-theatre-based sampling or web-based sampling on individual response. FindingsParticipants were predominantly women (1643 of 2583 [63·6%]), in first year (1079 of 2608 [41·4%]), and living in their parents' house (1085 of 2591 [41·9%]) or in a rented house or flat (1095 of 2591 [42·3%]). The median age of participants was 20 years (IQR 19–21). Responses differed significantly across lecture-theatre-based and web-based data collection methods by age, year in college, accommodation, physical activity, and alcohol consumption. Independent samples t tests showed alcohol consumption scores and mental health and wellbeing scores were lower among the lecture theatre sample (p=0·001). Controlling for all sociodemographic (age, sex) and university-level effects (faculty, accommodation type, and year in college), small but significant differences were still noted. Those who completed the web-based survey were 60% less likely to report high levels of physical activity (odds ratio 0·37, 95% CI 0·18–0·77; p=0·006) but twice as likely to have taken recreational drugs in the previous 12 months (1·82, 1·12–2·97; p=0·016). InterpretationStudents were sampled from the same third-level institution. The web-based approach provided an expected low response rate, compatible with previous research. It also displayed a different mixture of students in relation to age, year in college, and accommodation type to the lecture-based survey. After controlling for sociodemographic effects, differences in reporting of physical activity, alcohol use, and mental health and wellbeing remained. Although web-based approaches are more cost effective, small but significant differences were noted in reporting. Because of the large discrepancy in response rates (51% vs 2·4%), an absence of sample comparability was noted. A potential for selection bias is evident. Because data for student health behaviour are needed to inform student health policy and practice, further research in this area is needed to validate the effectiveness of either survey method. FundingNone.