A - 181 Test Fairness and True Peer Norms for English Learners: a Validation Study of the Ortiz Picture Vocabulary Acquisition Test.

Abstract

The Ortiz Picture Vocabulary Acquisition Test (Ortiz PVAT; Ortiz, 2018) incorporates English learners (ELs)' lifetime exposure to English (LEE) in the assessment of ELs' English receptive vocabulary. The current study aimed to examine the validity of the test and to provide support for the use of true peer norms that control for differences in LEE among ELs. An archival community-based dataset (N = 226; ages 2:7 and 18:11) drawn from the New York City metropolitan area was used to assess generalizability of Ortiz PVAT test scores, differences in standard scores obtained via traditional age norms vs. EL norms which account for LEE alongside age, and the predictive value of LEE. No significant differences in Ortiz PVAT performance was found based on gender or home language group membership, suggesting that the test measures receptive vocabulary in English in a fair manner across those demographic groups. Higher LEE was associated with better performance on the Ortiz PVAT. Significant differences in age-based standard scores with a medium effect size was found between monolingual English speakers and ELs with low or moderate LEE but not when LEE is accounted for using the EL norms. Lastly, an additional 18% of variance in Ortiz PVAT raw scores was accounted for by LEE above and beyond age, and LEE was found to exert more influence on test performance compared to age. Results added to the existing fairness evidence for the Ortiz PVAT and supported the use of true peer norms to account for differences in LEE.