Methods to increase response rates to postal questionnaires

Abstract

Postal questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response rates to postal questionnaires would improve the quality of health research. To identify effective strategies to increase response rates to postal questionnaires. We aimed to find all randomised controlled trials of strategies to increase response rates to postal questionnaires. We searched 14 electronic databases to February 2003 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, authors were also contacted to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria. Randomised controlled trials of methods to increase response rates to postal questionnaires. We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios and 95% confidence intervals in a random effects model. Evidence for selection bias was assessed using Egger's weighted regression method and Begg's rank correlation test and funnel plot. Heterogeneity among trial odds ratios was assessed using a chi-square test at a 5% significance level and the degree of inconsistency between trial results was quantified using I(2). We found 372 eligible trials. The trials evaluated 98 different ways of increasing response rates to postal questionnaires and for 62 of these the combined trials included over 1,000 participants. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.99, 95% CI 1.81 to 2.18; heterogeneity p<0.00001, I(2)=78%), recorded delivery (2.04, 1.60 to 2.61; p=0.0004, I(2)=69%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08, 1.27 to 7.44) and a more interesting questionnaire topic (2.44, 1.99 to 3.01; p=0.74, I(2)=0%). The odds of response were substantially higher with pre-notification (1.50, 1.29 to 1.74; p<0.00001, I(2)=90%), follow-up contact (1.44, 1.25 to 1.65; p<0.0001, I(2)=68%), unconditional incentives (1.61, 1.27 to 2.04; p<0.00001, I(2)=91%), shorter questionnaires (1.73, 1.47 to 2.03; p<0.00001, I(2)=93%), providing a second copy of the questionnaire at follow-up (1.51, 1.13 to 2.00; p<0.00001, I(2)=83%), mentioning an obligation to respond (1.61, 1.16 to 2.22; p=0.98, I(2)=0%) and university sponsorship (1.32, 1.13 to 1.54; p<0.00001, I(2)=83%). The odds of response were also increased with non-monetary incentives (1.13, 1.07 to 1.21; p<0.00001, I(2)=71%), personalised questionnaires (1.16, 1.07 to 1.26; p<0.00001, I(2)=67%), use of coloured as opposed to blue or black ink (1.39, 1.16 to 1.67), use of stamped return envelopes as opposed to franked return envelopes (1.29, 1.18 to 1.42; p<0.00001, I(2)=72%), an assurance of confidentiality (1.33, 1.24 to 1.42) and first class outward mailing (1.12, 1.02 to 1.23). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94, 0.88 to 1.00; p=0.51, I(2)=0%), when questionnaires began with the most general questions (0.80, 0.67 to 0.96), or when participants were offered the opportunity to opt out of the study (0.76, 0.65 to 0.89; p=0.46, I(2)=0%). Health researchers using postal questionnaires can increase response rates using the strategies shown to be effective in this systematic review.