Reading-Aloud Versus Self-Administered Student Questionnaires: An Experiment on Data Quality

Abstract

A major finding from recent large-scale assessments on student achievement is that a remarkable proportion of students around the world are poor readers. This calls into question the quality of the data retrieved from self-administered background questionnaires. A better administration mode, especially for this student population, might be to have the administrator read the questionnaires out aloud, as is done in surveys at elementary schools. In order to provide empirical evidence on whether reading aloud helps improve data quality, we conducted an experimental study with 664 twelve-year-old students in lower secondary schools in Germany. One finding is that, unsurprisingly, reading questionnaires aloud increases survey time. Regarding data quality, however, item non-response rates decrease somewhat in the reading- aloud group, and filtering procedures also work better. This effect can be found regardless of students’ status or reading speed. Even though the data quality for this group is generally poor, the improvement achieved by reading aloud is comparable with the group of fast readers. Regarding the acceptance of the mode, analyses on the role of migrant status and reading speed suggest that slow readers and migrant students particularly prefer being read the questionnaires aloud. Our study indicates that reading questionnaires aloud may be a meaningful administration mode not only in early primary school grades, but also at the beginning of secondary school. Data quality in studies involving at-risk students can particularly benefit from reading questionnaires aloud.