In-house or commercial speaking tests: Evaluating strengths for EAP placement

Abstract

When language program administrators consider changing a placement test, there are many issues to address. Will the scores help us place students into our curriculum? Will the scores reflect real differences in students' abilities? Will the administration of the test be feasible? This article describes one program's deliberations between keeping an in-house test or adopting a commercial test for speaking. Two speaking tests were compared according to curricular coverage, statistical distributions, and practicality. One test, PIE Speaking, was developed in-house. The other test, Versant English, was developed by Pearson Knowledge Technologies. Both covered many but not all curricular objectives. Internal consistency estimates were higher for Versant English than for PIE Speaking. The comparison of distribution patterns suggested that PIE Speaking better discriminated between mid-level students, but Versant English better discriminated between low and high ability students. PIE Speaking took approximately 60 staff hours, costing about $1200. Versant English took about 10 staff hours at an estimated cost of $6500. Cost weighed most heavily in the decision to keep the in-house speaking test. Modeling the steps taken to answer specific questions may provide structure for other language programs when evaluating their placement tests.