Data Quality of Different Modes of Supervision in Classroom Surveys

Abstract

Conducting quantitative research involving adolescents demands a thoughtful approach to the question of supervision, given that each option comes with its distinct set of implications. This study reviews these implications and empirically tests whether differences in data quality can be found among three modes of standardized survey research with medium-sized groups of adolescents (12–17 years). The data basis is a quasi-experimental survey study testing different forms of digital, hybrid, or in-person supervision that took place in 2021 in secondary schools in Germany (N = 923). The aim of this study is to test how aspects of data quality—item nonresponse, interview duration, drop-out rate, and response patterns—differ between these forms of supervision. Results could help researchers surveying young people to decide (1) whether they allow confidants or other adults to be present during interviews, (2) if they can rely on teachers alone when surveying classrooms, and (3) if it is cost-efficient to send out external supervisors for classroom sessions. While drop-out rates do not differ, item non-response, interview duration, and response patterns differ significantly; students supervised at home by external interviewers answered more questions, took more time to answer, and were less likely to give potentially meaningless answers in grid questions. The implications drawn from the findings question the common approach of solely relying on teachers for survey administration without the support of external supervisors or adequate training. Recruiting respondents via schools and surveying them online in their homes during school hours has been shown to be a robust method with regard to the analyzed indicators.