Native Speaker Raters Research Articles

Lexical aspects of comprehensibility and nativeness from the perspective of native-speaking English raters

Abstract This study analyzed the contribution of lexical factors to native-speaking raters’ assessments of comprehensibility and nativeness in second language (L2) speech. Using transcribed samples to reduce non-lexical sources of bias, 10 naïve L1 English raters evaluated speech samples from 97 L2 English learners across two tasks (picture description and TOEFL integrated). Subsequently, the 194 transcripts were analyzed through statistical software (e.g., Coh-metrix, VocabProfile) for 29 variables spanning various lexical dimensions. For the picture description task, separation in lexical correlates of the two constructs was found, with distinct lexical measures tied to comprehensibility and nativeness. In the TOEFL integrated task, comprehensibility and nativeness were largely indistinguishable, with identical sets of lexical variables, covering dimensions of diversity and range. Findings are discussed in relation to the acquisition, assessment, and teaching of lexical properties in L2 speech.

Promoting Intercultural Competence of Thai University Students through Role-Play

The current situation of international communication in globalization context requires intercultural competence (IC) to achieve successful communication (Crystal, 2003). Concerning this intercultural competence, non-verbal communication (NVC) plays a key role to indicate the success of having intercultural competence. On the other hand, Thai Ministry of Education (MOE) has launched the CEFR for the country policy in English teaching and learning (Ministry of Education, 2016). The purpose of this paper is to present major results of an investigation of intercultural competence, that is, the NVC. An innovative program implemented in this study is the CEFR and project- based activities (the CEFR-PBA). The study also examines Thai university students’ attitude towards this innovative class. This paper discusses the IC students learned from role-plays. The participants of the study were 44 students of third year English Education at a Thai Rajabhat University. Research method applies action research (Burns, 1999) and case study (Yin, Case Study Research: Design and Method, 2003). Data collection gained was from 1) videotape recordings of students’ role-play, and 2) the questionnaire. Data analysis for video recordings employed a NVC rating scale evaluated by native-speaker raters; while means, percentage, and SD were used for the questionnaire. Grounded theory’s color coding (Strauss & Cobin, 1990) will be utilized in analyzing the qualitative data from raters’ additional comment. Findings revealed students’ NVC performance were both appropriate on body movement and position as well as vocalic communication. Moreover, their attitude towards the CEFR-PBA was positive.

LEXICAL PROFILES OF COMPREHENSIBLE SECOND LANGUAGE SPEECH

This study examined contributions of lexical factors to native-speaking raters’ assessments of comprehensibility (ease of understanding) of second language (L2) speech. Extemporaneous oral narratives elicited from 40 French speakers of L2 English were transcribed and evaluated for comprehensibility by 10 raters. Subsequently, the samples were analyzed for 12 lexical variables targeting diverse domains of lexical usage (appropriateness, fluency, variation, sophistication, abstractness, and sense relations). For beginner-to-intermediate speakers, comprehensibility was related to basic uses of L2 vocabulary (fluent and accurate use of concrete words). For intermediate-to-advanced speakers, comprehensibility was linked to sophisticated uses of L2 lexis (morphologically accurate use of complex, less familiar, polysemous words). These findings, which highlight complex associations between lexical variables and L2 comprehensibility, suggest that improving comprehensibility requires attention to multiple lexical domains of L2 performance.

Lexical correlates of comprehensibility versus accentedness in second language speech

The current project investigated the extent to which several lexical aspects of second language (L2) speech – appropriateness, fluency, variation, sophistication, abstractness, sense relations – interact to influence native speakers’ judgements of comprehensibility (ease of understanding) and accentedness (linguistic nativelikeness). Extemporaneous speech elicited from 40 French speakers of English with varied L2 proficiency levels was first evaluated by 10 native-speaking raters for comprehensibility and accentedness. Subsequently, the dataset was transcribed and analyzed for 12 lexical factors. Various lexical properties of L2 speech were found to be associated with L2 comprehensibility, and especially lexical accuracy (lemma appropriateness) and complexity (polysemy), indicating that these lexical variables are associated with successful L2 communication. In contrast, native speakers’ accent judgements seemed to be linked to surface-level details of lexical content (abstractness) and form (variation, morphological accuracy) rather than to its conceptual and contextual details (e.g., lemma appropriateness, polysemy).

IPMDS-Sponsored Scale Translation Program: Process, Format, and Clinimetric Testing Plan for the MDS-UPDRS and UDysRS.

We present the methodology and results of the clinimetric testing program for non-English translations of International Parkinson and Movement Disorder Society (MDS)-sponsored scales. The programs focus on the MDS revision of the UPDRS (MDS-UPDRS) and the Unified Dyskinesia Rating Scale (UDysRS). The original development teams of both the MDS-UPDRS and UDysRS envisioned official non-English translations and instituted a rigorous translation methodology. The formal process includes five core steps: (1) registration and start-up; (2) translation and independent back-translation; (3) cognitive pretesting to establish that the translation is clear and that it is comfortably administered to and completed by native-speaker raters and patients; (4) field testing in the native language using a large sample of Parkinson's disease patients; and (5) full clinimetric testing. To date, the MDS-UPDRS has 21 active language programs. Nine official translations are available, having completed all phases successfully, and the others are in different stages of development. For the UDysRS, 19 programs are active, with three official translations now available and the rest in development at different stages. Very few scales in neurology and none in movement disorders have fully validated translations, and this model may be adopted or modified by other scale programs to allow careful validation of translations. Having validated translations allows for maximal homogeneity of tools utilized in multicenter research or clinical trial programs.

Incidental Development of L2 Proficiency in NS-NNS Email Interactions

Recent research suggests that email can be a powerful motivator for authentic L2 interaction, but little is known about the efficacy of this medium in the development of target language proficiency. The present study addresses this issue by examining email exchanges between university learners of Japanese as a foreign language and native Japanese university students. Of interest is the effect of email interactions on the incidental development L2 syntax and vocabulary as reflected in both quantitative and qualitative measures. Messages sampled at regular intervals over a 5-week collection period indicated a reliable increase in syntactic development as reflected in several measures of structural mastery as well as in qualitative ratings supplied by native speaking raters. There was no evidence of quantitative development for vocabulary, but qualitative ratings did show a small improvement over the collection period. A sharp drop-off was noted between the first and the second samples across all measures, with learner performance then improving steadily till the end of the study. The findings are related to an interactionist account of L2 development that is embedded in the framework of computer-mediated communication (CMC). Issues in research methodology are also discussed.

Rater variation in the assessment of speech acts

This study addresses variability among native speaker raters who evaluated pragmatic performance of learners of English as a foreign language. Using a five-point rating scale, four native English speakers of mixed cultural background (one African American, one Asian American, and two Australians) assessed the appropriateness of two types of speech acts (requests and opinions) produced by 48 Japanese EFL students. To explore norms and the reasoning behind the raters’ assessment practice, individual introspective verbal interviews were conducted. Eight students’ speech act productions (64 speech acts in total) were selected randomly, and the raters were asked to rate each speech act and then explain their rating decision. Interview data revealed similarities and differences in their use of pragmatic norms and social rules in evaluating appropriateness.

English Immersion in Paraguay: Individual and Sociocultural Dimensions of Language Learning and Use

This investigation of English immersion learners explored processes of learning English, patterns of language use, and levels of comprehensibility among 34 predominantly Spanish-speaking 12th graders at the American School of Asuncion in Asuncion, Paraguay. The analysis of data from language learning histories, group interviews, perceived comprehensibility ratings, and questionnaires about language use led to the identification of linguistic dimensions affecting comprehensibility and of individual and sociocultural dimensions related to language learning and use. When judging comprehensibility, the native-speaker raters were influenced mainly by prosodic elements and overall fluency. Higher comprehensibility corresponded to girls as well as to students with some schooling in English language contexts, aspects of integrative motivation and more frequent L2 use. Findings were congruent with studies in immersion education, second language acquisition, sociolinguistics and social psychology.

Second-Language Fluency and Person Perception in China and the United States

The hypothesis that fluency of non-native speakers would affect how they were perceived by native speakers was examined. Speech samples were first collected from non-native speakers of Chinese and from non-native speakers of English. Each language sample was then rated for fluency, after which 110 native Chinese speakers and 114 native American English speakers rated the non-native speakers of the respective languages on seven personality scales. A multivariate analysis showed that more fluent non-native speakers of either language were perceived more positively by native speaking raters than less fluent speakers. Ratings were influenced by the order in which raters heard speakers. This was manifested by a main effect for order among American raters, and an interaction effect with fluency for both groups. The fluency of initially heard speakers appeared to establish a standard against which subsequently heard speakers were judged. These findings were observed using personality descriptors which have similar connotative and denotative meaning across the two languages, allowing comparability of results. Although findings were very similar for both groups, increases in fluency affected ratings more uniformly for Chinese than for American raters. Implications for contextual influences on person perception and social cognition are discussed.