CONSIDERATIONS AND TECHNIQUES FOR THE ANALYSIS OF NAEP DATA

Abstract

ABSTRACTThe special characteristics of the National Assessment data which affect the validity of conventional techniques of statistical analysis are considered. In contrast to the assumptions underlying standard methods of statistical analysis, the NAEP samples are obtained via a stratified multi‐stage probability sampling design in which clusters of students are selected and in which certain subpopulations are sampled at a higher rate. The resulting samples have different statistical characteristics from those of a simple random sample. Analytic techniques which take the structure of the data into account are discussed and their properties explored. A second feature of the National Assessment data arises from the fact that multiple matrix sampling techniques are used for the assignment of subsets of the pool of exercises to subsamples of students. National Assessment accounts for the effect of the missing data on its scaling models through the use of multiple imputation procedures; creating vectors of “plausible values” for the scale‐scores. The use of plausible values and their effect on analysis are illustrated. The analyses are illustrated with results from the NAEP assessments of mathematics.