Paper-and-pencil versus computerized administration mode: Comparison of data quality and risk behavior prevalence estimates in the European school Survey Project on Alcohol and other Drugs (ESPAD).

Emanuela Colasante,Roberta Potente,Arianna Cutilli,Elisa Benedetti,Sabrina Molinaro,Marco Scalese,Loredana Fortunato,Hajo Zeeb

doi:10.1371/journal.pone.0225140

Abstract

PurposeThe aim of this experimental study was to investigate whether paper-and-pencil and computerized surveys administered in the school setting yield equivalent data quality indicators and risk behavior prevalence estimates.MethodsData were drawn from the European School Survey Project on Alcohol and Other Drugs (ESPAD®) carried out in Italy to monitor drug, alcohol, tobacco use and other risk-behaviors among Italian high school students aged 15–19 years.A sub-sample of schools was recruited for the study (1673 pupils). For each school, two entire randomly selected courses (from the first to the fifth grade) participated and were assigned randomly to the self-administered paper-and-pencil (N = 811) or computerized survey (N = 862). Differences in data quality were assessed using the following indicators: questionnaire completeness (missing gender and/or 50% of missing answers) and internal consistency (repetitive extreme response patterns). Separate logistic regression models were used to estimate the mode effect on the reporting of each risk behavior, controlling for gender and age. Finally, the prevalence estimates of the experimental study were compared to the results of the national ESPAD® study.ResultsThe computerized administration mode produced a higher proportion of invalid questionnaires, but the prevalence estimates generated from responses to the paper-and-pencil and computerized surveys were generally equivalent. Nevertheless, comparing these results with those of the national ESPAD® study, some differences in the prevalence rates were found.ConclusionsThe findings suggest that in a proctored school setting, the computerized survey mode yields almost the same results as the paper-and-pencil mode. However, because of the reliance on existing informatics facilities until when all schools in the country will be sufficiently equipped for the computerized data collection, they should be given the opportunity to choose between paper-and-pencil and computerized survey modes, in order to avoid a possible selection bias.

Highlights

Over the past 20 years, there has been an increasing use of computerized surveys for data collection in health and social science research
The computerized administration mode produced a higher proportion of invalid questionnaires, but the prevalence estimates generated from responses to the paper-and-pencil and computerized surveys were generally equivalent
The findings suggest that in a proctored school setting, the computerized survey mode yields almost the same results as the paper-and-pencil mode

Summary

Introduction

Over the past 20 years, there has been an increasing use of computerized surveys for data collection in health and social science research The advantages of this kind of administration mode over self-administered paper-and-pencil (P&P) questionnaires include, in particular, the reduced data-collection costs, the avoidance of data entry errors and the accessibility of data in a short time. These advantages are contributing to a greater consideration in the management of cross-sectional studies that use self-administered questionnaires, leading to the progressive replacement of the P&P method with the computerized testing [1,2]. In the case of adolescents’ self-reports of sensitive behaviors, it is generally assumed that self-administration leads to increased disclosure of sensitive and socially undesirable behaviors as compared to interviewer-administered surveys [3,4,9,10], other factors could play a role in misreporting, such as the setting (e.g. the presence of others while responding) and being assured about the anonymity [7]

Methods

Results

Conclusion