Linking Statewide Tests to the National Assessment of Educational Progress: Accuracy of Combining Test Results Across States

Abstract

The National Assessment of Educational Progress (NAEP) surveys achievement at selected grades and content areas and does not report scores at the individual, school, or district level. There is a desire by schools and school districts to compare results of their own assessments to the national results provided by NAEP. The main objective of this study is to investigate the accuracy of linking NAEP scores to statewide test results. This study investigates whether the population invariance condition of the function for linking two sets of scores holds, specifically whether the function obtained for an individual state is the same as the function obtained for other states either individually or combined. Using an equipercentile procedure, functions obtained separately for four states are compared to a function obtained using data combined across the four states. The results suggest that the link between statewide tests and the NAEP does not provide precise information and that the information from a linking study such as this one should be limited to rough estimates of percentages of students in each of the NAEP achievement levels. Two areas of concern are identified: (a) the differences between the statewide tests and the NAEP test and (b) the error due to results from the two sets of tests and the error due to the linking analyses.