Question Order and Interviewer Effects in CATI Scale-up Surveys

Abstract

The scale-up estimator is a network-based estimator for the size of hidden or hard to count subpopulations. Several issues arise in the public health context when the aim is the estimation of injuries occurring in a certain population, where two common problems are present: (a) Small injuries are usually difficult to observe and rarely reported in the official data and (b) people are not always compliant in giving information about some specific injuries, in particular when children are involved. This study checked the methodological issues arising from using a computer-assisted telephone interview (CATI) survey using the scale-up methodology for detecting the number of injuries due to choking in children ages 0 to 14 in Italy. For this purpose, 1,000 CATI interviews were conducted during a week using a questionnaire based on 33 questions about populations of known size according to census data. Then, each respondent was asked about other questions related to the main target population (e.g., number of children known to suffer from a choking accident). A sensitivity analysis was conducted for estimating the effect of varying subpopulations, order of the questions, and interviewer effects on the resulting estimates. For the interviewer effect, no particular differences were observed in the overall estimates of injuries. The conclusion is the scale-up estimator in association with CATI methodology shows a high potential in the field of injury prevention, being accurate and robust, but particular attention should be given to the training of the interviewers to improve stability of the estimates.