Validity and reliability of responses to a self-report home safety survey designed for use in a community-based child injury prevention programme

Abstract

Objective: To quantify the validity and reliability of responses to a self-report home safety survey designed for use in a community-based child injury prevention program. Methods: A cross-sectional study of home-based injury risk factors and safety practices was administered to a random sample of households (n = 614) in two rural and remote communities in Queensland, Australia using a Computer Assisted Telephone Interview (CATI) survey. The survey was re-administered during subsequent home interviews with a sub-sample (n = 85) of CATI participants. Results: For all self-reported hazards and safety practices, prevalence estimates obtained from the CATI survey were significantly different from those directly observed at the home interview with proportions of homes exposed to hazards between 10–21% under reported at CATI survey and safety practices over-reported at CATI survey by 17–24%. There was no statistically significant difference however between the estimates of prevalence of hazards and safety practices self-reported at CATI and home interview. Validity of the CATI survey was poor with sensitivity of the CATI question ranging from 0 to 71 and specificity from 32 to 97. While the marginal distribution of prevalences of hazard prevalences and safety practices were similar between CATI and home interviews, the low level of response concordance (κ = −0.24 to 0.41) indicates that reliability of the responses to these questions was low. Conclusion: In view of these limitations in the accuracy of CATI risk factor surveys, their use for priority setting and evaluation in community-based injury prevention programs needs to be considered with caution.