AN EXPLORATION OF SPEAKING PROFICIENCY MEASURES IN THE TOEFL CONTEXT

Abstract

ABSTRACTThis report describes a three‐year study involving the initial development and experimental administration of a variety of test formats and item types aimed at measuring the English speaking proficiency of TOEFL candidates or other non‐native speakers of English on a direct basis, i.e., through the evaluation of actual speech production on the part of the examinee.Although practical constraints of the TOEFL program ruled out of consideration, for operational purposes, face‐to‐face testing procedures of the Foreign Service Institute (FSI) type, a variety of “semi‐direct” testing formats (involving test stimuli delivered by tape recording and accompanying booklet, with examinee speech tape recorded for evaluation) were developed as part of the study. These were administered to undergraduate and graduate level non‐native English speaking students on a small‐scale clinical basis and, following revisions, to a fairly large (N = 155) examinee group. An FSI‐type interview was also administered to all study participants as an external criterion measure of speaking proficiency.Correlational and scalar analyses were used to identify subsets of item formats and individual items having satisfactory correlations with the FSI criterion interview and also meeting relevant linguistic and administrative requirements. These were grouped into a prototype “Test of Spoken English” (TSE) that was found to increase predictability of total FSI variance by approximately 8% when added to score data provided by the regular (paper‐and‐pencil) TOEFL. Further modifications in TSE content and follow‐up administration with larger and more heterogeneous candidate groups are expected to increase this value.Two additional related studies are currently under way, involving pilot test administration at regular TOEFL sites and use‐validation of the TSE as a measure of the “classroom communication” effectiveness of non‐native English speaking teaching assistants in U.S. academic settings.