Where nonresponse is at its loudest: Cross-country and individual differences in item nonresponse across the PISA 2018 student questionnaire

Abstract

The Programme for International Student Assessment (PISA) student questionnaire, despite being designed for low cognitive demand, may induce test burden due to its 306-item length, resulting in increased item nonresponse toward the questionnaire’s end. Using the PISA 2018 response data from 80 countries and a cross-classified mixed effects model, this study explored the average, cross-country, and individual nonresponse trends throughout the questionnaire. The average nonresponse probability increased from 2% at the questionnaire’s start to 11% at its end. East Asian countries displayed the flattest nonresponse trends, while South American countries showed the steepest. Individual differences were also prominent: students with a lower initial nonresponse propensity experienced a steeper increase in their nonresponse propensity as they progressed through the questionnaire, whereas those with a higher initial propensity remained at a relatively constant level. Our findings highlight the importance of the questionnaire design, the context in which it is administered, and its interaction with individual and cultural factors in shaping student response behavior. We discuss the implications for questionnaire design and researchers using PISA student questionnaire data for secondary analysis.