What Do Test-Takers Say? Test-Taker Feedback as Input for Quality Management of a Local Oral English Proficiency Test

Abstract

The Oral English Proficiency Test (OEPT) is a computer-administered, semi-direct test of oral English proficiency used to screen the oral English proficiency of prospective international teaching assistants (ITAs) at Purdue University. This paper reports on information gathered from the post-test questionnaire (PTQ), which is completed by all examinees who take the OEPT. PTQ data are used to monitor access to the OEPT orientation video and practice test, to evaluate examinee perceptions of OEPT characteristics and administration, and to identify any problems examinees may encounter during test administration. Responses to the PTQ are examined after each test administration (1) to ensure that no undue or unexpected difficulties are encountered by examinees and (2) to provide a basis for modifications to our administrative procedures when necessary. In this study, we analyzed 1440 responses to both closed-ended and open-ended questions of the PTQ from 1342 test-takers who took the OEPT between August 2009 and July 2012. Responses to these open-ended questions on the OEPT PTQ provided an opportunity to examine an unexpectedly wide variety of response categories. The analysis of the 3-year data set of open-ended items allowed us to better identify and evaluate the effectiveness of changes we had introduced to the test administration process during that same period of time. Carefully considering these responses has contributed substantially to our quality control processes.