Consistency in reasoning test scores over time

Abstract

UK schools have a long history of using reasoning tests, most frequently of Verbal Reasoning (VR), Non-Verbal Reasoning (NVR), and to a lesser extent Quantitative Reasoning (QR). Results are used for identifying students' learning needs, for grouping students, for identifying underachievement, and for providing indicators of future academic performance. Despite this widespread use there are little empirical data on the long-term consistency of VR, QR and NVR as discrete abilities. To evaluate and compare the consistency of VR, QR and NVR scores over a 3-year period, and to explore the influence of the secondary school on pupils' progress in the tests. Data were collected on a longitudinal sample of over 10,000 pupils who completed the Cognitive Abilities Test Second Edition in year 6 (age 10+) and year 9 (age 13+), and GCSE public examinations in year 11 (age 15+). Correlation coefficients and change scores for individual pupils are calculated. Multilevel modelling is used to determine school effects on reasoning scores and GCSE public examination results. The results reveal high correlations in scores over time, ranging from .87 for VR to .76 for NVR, but also show around one-sixth of pupils on the VR test and one-fifth of pupils on the QR and NVR tests change their scores by 10 or more standard score points. Schools account for only a small part of the total variation in reasoning score, although they account for a much greater proportion of the variation in measures of attainment such as GCSE. School effects on pupils' progress in the reasoning tests between age 10 and age 13 are relatively modest. Reasoning tests make excellent baseline assessments for secondary schools. Some practical and policy implications for schools are discussed.