Post-collection Data Capture, Scoring, and Processing

Abstract

The International Association for the Evaluation of Educational Achievement (IEA) collects large-scale assessment data from multiple countries and in many languages. Three important processes are data capture, scoring and coding, and data processing, which are supported by rigorous quality control mechanisms that IEA uses to ensure data comparability across countries. This chapter describes the steps that IEA takes to ensure the reliability of the data. Depending on the scope of the study, IEA uses different data capture systems during or after data collection. While data from multiple choice items can be captured directly, other data (e.g., responses from open ended questions) must be coded or scored uniformly to ensure comparability across countries and safeguard quality. Data processing starts only after data capture, scoring, and coding of data are finalized. As part of its data quality measures, IEA provides training for national representatives, instructional manuals, and study-specific software products to participating countries. Check protocols included in the software facilitate adherence to technical standards and minimize errors. Some data, while collected according to national conventions, ultimately needs to conform to IEA international formats. Moreover, the data must be assembled in a way that enables three linkages: schools with countries, teachers and classes with schools, and students with their teachers and parents. Data processing also detects and recodes any deviations from international formats or insufficient links between respondent levels.