A Comparison of the Effects of Analytic and Holistic Rating Scale Types in the Context of Composition Tests

Abstract

This study examines how different composition rating scale types—analytic and holistic—can differentially affect the aspects of academic English ability measured in an ESL proficiency test battery. Specifically, the study addresses the following questions: (1) To what extent do holistic and analytic scales contribute differentially to total scores on a test of academic English ability? (2) To what extent does the test as a whole measure different aspects of language ability, depending on whether analytic or holistic composition scores are used? (3) To what extent does a particular rating scale type provide potentially useful information for placement or diagnosis, either alone or as part of a multi-component assessment? Multiple regression and exploratory factor analyses indicate that changing the composition rating scale type not only changes the interpretation of that section of a test, but may also result in total test scores which are no longer comparable.