Changing times, changing needs: enhancing the utility of international large-scale assessments

Abstract

Mounting concerns about the levels and distributions of human capital, as well as how they are associated with outcomes for individuals and societies, have contributed to an increase in the number of national and international surveys. These surveys not only examine skills among school-age and adult populations, they also facilitate evaluation of the relationships among these skills and various background factors. At this juncture, the main ILSAs are making the transition to becoming fully digitally based assessments (DBAs). With the transition rapidly progressing, this is a propitious moment to consider the history of large-scale national and international assessments and to reflect on both what has contributed to their increased salience and growth, and how best to enhance their constructive impact on both policy and policy research in the future. We argue this can be done by utilizing a comprehensive, multidimensional framework that establishes a set of design criteria against which these assessments, present and future can be evaluated. The original framework was proposed more than 30 years ago by Messick (European Journal of Psychology of Education II 2:157–165, 1987) when all large-scale assessments were developed and delivered with paper-based instruments. Messick’s framework provided an organizational structure for how to think about and evaluate the potential utility of all large-scale assessments. After presenting a brief historical perspective on the development and growth of large-scale assessments, we review and extend Messick’s original framework. We then describe how the transition to DBAs can impact each of the framework’s three criteria. We also offer recommendations on how ILSAs’ development and innovations can best be deployed so that they are of greater utility to policy makers and other stakeholders worldwide.