An investigation of the validity of a speaking assessment for adolescent English language learners

Abstract

Given the increasing emphasis of communicative competence in English as a foreign language (EFL) contexts and the lack of validation research on speaking assessments for adolescent EFL learners, in the current study we examined the validity of the TOEFL Junior® speaking test, a relatively new speaking assessment developed by Educational Testing Service. We utilized Messick’s integrative view of construct validity (1996) and Kane’s interpretation argument framework (Kane, 2013), and focused on two sources of validity evidence: internal structure and external relations to other variables (AERA, APA, and NCME, 2014). We used a cross-sectional design with 252 seventh-to-ninth graders in Taiwan, which aligns with the spectrum of the TOEFL Junior®’s target age group. All adolescent participants took the TOEFL Junior® speaking test and two researcher-developed speaking tests. They also completed a survey reporting their history of English learning experiences from kindergarten to present and evaluating their own English language skills. Adolescent participants’ teachers provided evaluations of participants’ English language skills. Results from correlational tests and confirmatory factor analysis showed robust evidence for the test’s internal structure. Correlation and multiple regression models also demonstrated strong positive relationships between TOEFL Junior® test scores and external variables, providing evidence for TOEFL Junior®’s construct validity.