Using Statistical Transformation Methods to Explore Speech Perception Scale Lengths

Abstract

The practice of second language (L2) speech perception has traditionally relied on equal-interval perceptual scales and novice listeners’ (NLs) impressionistic judgments of constructs such as accentedness and comprehensibility (Munro & Derwing, 2011). However, issues have surfaced with respect to how well NLs can use these scales, whether they use the entire scale, and how valid/reliable their ratings are. This study draws inspiration from the work of Glenn Fulcher in the area of L2 speech assessment, specifically on construct validity and scale design (1993; 2003). The current study explores similar issues from the lens of the speech perception practice, relying on scale transformation methods and multi-faceted Rasch measurement to examine the overall question of an ideal equal-interval scale length. Speech ratings of L2 accentedness and comprehensibility were originally collected from 56 NLs on 9-point bipolar Likert scales. Statistical transformations from 9-point to 7- and 5-point scales were compared through several indices (i.e., listener consistency, listener severity, scalar point discrimination, and scale usage) across four common speech tasks (i.e., read-aloud, spontaneous speech, elicited imitation, and picture description) (see Thomson & Derwing, 2015). With every statistical scale length reduction across four tasks, listeners’ consistency improved, their levels of severity slightly decreased, and their scalar usage became more precise. Implications for L2 speech perception research suggest that the shorter 5-point perceptual scale shows potential in increasing the reliability and validity of scoring, while more appropriately distinguishing among speaker ability levels.