GENERALIZATION OF SAT® VALIDITY ACROSS COLLEGES

Abstract

ABSTRACTThis study, which focused on the validity of the SAT‐V and SAT‐M, used data from 99 validity studies that were conducted by the Validity Study Service of the College Board. In addition to test validities based on first‐year college averages, which were calculated using institutional data, validities for each college were also estimated for two other groups—applicants for admission to the colleges, and all SAT takers. These last two estimates were based on range restriction theory.Substantial validity generalization was found: the assumption that applicant pool validities were all equal, together with sampling variance and the effects of selection, accounted for 36 percent and 34 percent of the variation of the SAT‐V and SAT‐M validities, respectively. The hypothesis of equal validity in pools like those of all SAT takers, plus sampling variance and the effects of selection, accounted for 53 percent and 33 percent of the variation of the SAT‐verbal and SAT‐mathematical validities, respectively. However, significant institutional uniqueness remains, though part of that uniqueness consists of variation in the reliability of first‐year college average.For these data, substantial validity was the rule. The average validities were quite high, rising to .55 for either SAT‐V or SAT‐M true scores for all SAT takers, and 95 percent of the observed validities were above .13 for SAT‐V and .10 for SAT‐M. Values below these may be owing to accidents of sampling, computing errors, or criterion defects, and it should be noted that 95 percent is a conservative standard. Studies with slightly higher validities may be questioned as well, perhaps repeated, and the criterion examined carefully.A hypothesis that validities for SAT‐V and SAT‐M might differ across institutions but have the same ratio was also tested. It was thought that departures from this second assumption might lead to the detection of institutional types. However, significant departures from the equal‐ratio hypothesis tested at the 5 percent level occurred for about 5 percent of the institutions, so no detection of institutional types occurred.