THE TEST OF SPOKEN ENGLISH AS A MEASURE OF COMMUNICATIVE ABILITY IN THE HEALTH PROFESSIONS: VALIDATION AND STANDARD SETTING

Abstract

ABSTRACTThe ability to communicate effectively is considered critical to successful performance in numerous occupations, including many health‐related professions. Oral proficiency is necessary in some degree for performing such tasks as interacting with medical colleagues and counseling or instructing patients. The importance of communicative competence is evidenced by a growing number of states' and certification agencies' requirements for minimum oral proficiency in a variety of occupations.The increasing interest in oral proficiency during the past decade prompted Educational Testing Service (ETS) to undertake the development of the Test of Spoken English, a standardized test of speaking proficiency of nonnative speakers of English. The test has been validated for the selection of nonnative teaching assistants applying to U.S. institutions.The objectives of the study reported here were to: (a) provide additional validation of the TSE for the selection and certification of nonnative health professionals, (b) establish a set of procedures for determining standards of proficiency on the TSE, and (c) try out these procedures in selected health professions in which the test could be used.Groups of judges who were practitioners in four health professions–nursing, pharmacy, medicine, and veterinary medicine–judged the acceptability of the speaking performance of examinees who had taken the TSE. Each group was asked to provide, for its profession, ratings for three distinct situations, such as communicating with professional colleagues, teaching in the particular field, or working in a hospital. Judges were asked to indicate whether each examinee was at least minimally acceptable to function in each situation. Judges' ratings were then related to the examinees TSE scores. A larger group of consumers of medical care gave more global ratings to the same examinees for each of the four professions.Background data suggested that the groups of judges were heterogeneous with regard to a number of relevant personal characteristics and experiences, such as degree of exposure to nonnative speakers. This diversity suggests that judges represented a variety of points of view about necessary language proficiency.Analyses of judges' ratings revealed a substantial degree of consistency. Even though only relatively small segments of test performance were rated, judges tended to assign the same ratings when asked to rate a second sample of test performance for the same examinees. Although the data showed that the procedures applied can sometimes produce different results from one occasion to the next, the relationships of judges' ratings to TSE scores generally exhibited considerable stability.Alternative methods were applied to determine possible ranges of cutoff scores on the TSE. One of these methods involved a consideration of the consequences of different decision outcomes, e.g., whether the certification or licensing of an unacceptable speaker is more or less serious than failing to certify an acceptable speaker. A range of possible cutoff scores was computed for each situation within each profession.The results of the study support the validity of the TSE as a measure of oral language proficiency and provide some progress toward the setting of appropriate standards of oral language proficiency within the health‐related professions.