Enhancing the Validity and Usefulness of Large-Scale Educational Assessments: II. NELS:88 Science Achievement

Abstract

This study is second in a series demonstrating that achievement tests are multidimensional and that using psychologically meaningful subscores in national educational surveys can enhance test validity and usefulness. NELS:88 8th- and l0th-grade science tests were subjected to full information item factor analysis. Factors reflecting everyday knowledge, scientific reasoning, chemistry knowledge, and reasoning with knowledge were obtained in 8th grade. Quantitative science, spatial-mechanical, and basic knowledge and reasoning were distinguishable factors in 10th grade. Regression analyses showed that different patterns of prior math and science achievement, and of course taking, were associated with each l0th-grade science factor. Teacher emphasis on problem solving and understanding related more to quantitative science, and basic knowledge and reasoning. Spatial-mechanical reasoning showed the strongest gender and ethnicity effects; it related also to science museum visits but not to instructional variables. It is recommended that multidimensional achievement scores be used to capture student and teacher effects that total scores used alone miss.