INVESTIGATING THE CRITERION‐RELATED VALIDITY OF THE TOEFL® SPEAKING SCORES FOR ITA SCREENING AND SETTING STANDARDS FOR ITAS

Abstract

ABSTRACTAlthough the primary use of the speaking section of the Test of English as a Foreign Language™ Internet‐based test (TOEFL® iBT Speaking test) is to inform admissions decisions at English medium universities, it may also be useful as an initial screening measure for international teaching assistants (ITAs). This study provides criterion‐related validity evidence for the use of TOEFL iBT Speaking for ITA screening and evaluates the effectiveness of using the scores for teaching assistantship (TA) assignment classification.Four universities participated in this study. Local ITA‐screening tests or instructor recommendations were used as the criterion measures. Relationships between the TOEFL Speaking test and the local ITA tests were explored through observed and disattenuated correlations. These relationships were moderately strong, supporting the use of the TOEFL Speaking test for ITA screening. However, the strengths of the relationship between the TOEFL Speaking test and the local ITA tests were found to be somewhat different across universities depending on the extent to which the local test engaged and evaluated nonlanguage abilities. Implications of these findings are discussed.Binary and ordinal logistic regressions were used to investigate how effective TOEFL Speaking scores were in separating students into distinct TA assignment categories. At all four universities, TOEFL Speaking scores were significant predictors of students' TA assignments and were fairly accurate in classifying students for TA assignments. ROC curves were used to determine TOEFL Speaking cut scores for TA assignments at each university that would minimize false positives (i.e., true nonpasses classified as passes).The results have considerable potential value in providing guidance on using the TOEFL iBT Speaking scores for ITA screening.