Language Proficiency Interviews Research Articles

Validity of the American Sign Language Discrimination Test

American Sign Language (ASL) is one of the most commonly taught languages in North America. Yet, few assessment instruments for ASL proficiency have been developed, none of which have adequately demonstrated validity. We propose that the American Sign Language Discrimination Test (ASL-DT), a recently developed measure of learners’ ability to discriminate phonological and morphophonological contrasts in ASL, provides an objective overall measure of ASL proficiency. In this study, the ASL-DT was administered to 194 participants at beginning, intermediate, and high levels of ASL proficiency, a subset of which ( N = 57) also was administered the Sign Language Proficiency Interview (SLPI), a widely used subjective proficiency measure. Using Rasch analysis to model ASL-DT item difficulty and person ability, we tested the ability of the ASL-DT Rasch measure to detect participant proficiency group mean differences and compared its discriminant performance to the SLPI ratings for classifying individuals into their pre-assigned proficiency groups using resource operating characteristic statistics. The ASL-DT Rasch measure outperformed the SLPI ratings, indicating that the ASL-DT may provide a valid objective measure of overall ASL proficiency. As such, the ASL-DT Rasch measure may provide a useful complement to measures such as the SLPI in comprehensive sign language assessment programs.

Analysis of sign language proficiency among teachers of the deaf in primary schools in Bulawayo (Zimbabwe): Implications for learning and inclusion

Sign Language proficiency among teachers of the deaf form the basis for effective learning by deaf children. It is also the bedrock of successful inclusion. Previous studies and literature have equivocally confirmed that young children who are deaf learn more effectively when taught using Sign Language. The current study sought to establish the level of Sign language competency among teachers of children who are deaf in 10 of the primary schools in Bulawayo that enroll children who are deaf. A descriptive survey and ex-post facto research designs were employed in soliciting data from a sample of 15 teachers of the deaf and 5 college lecturers. Data were collected using a structured questionnaire and a Modified Sign Language Proficiency Interview (MSLPI) on a 6-Point rating scale. Document analysis and observation were used to authenticate responses from college lecturers and teachers of the deaf respectively. The Statistical Package for Social Sciences (SPSS) and descriptive summaries were used to analyse data. The data indicated that teachers of the deaf in Bulawayo lacked Sign Language proficiency. They could only afford the lowest two scores on the (MSLPI) scale that is the Novice and the No Functional Skills levels. What teachers at times mistook for Sign language was mere finger spelling and some distorted signed systems. From these findings the study concluded that the learning of the deaf children in primary schools in Bulawayo was heavily compromised due to lack of effective communication between teachers and the learners. The study also concluded that the inclusion of deaf children in the schools would remain a pipe dream as long as the teachers were not proficient in Sign language which is the first language of the deaf. The study recommended regular Sign Language staff development programmes, intensification of the practical component of Sign Language in the college programme for trainee teachers of the deaf, external scholarship and exchange programmes for both teachers and college lecturers who train teachers for the deaf.

Determining aspects of text difficulty for the Sign Language of the Netherlands (NGT) Functional Assessment instrument

In this paper we describe our work in progress on the development of a set of criteria to predict text difficulty in Sign Language of the Netherlands (NGT). These texts are used in a four year bachelor program, which is being brought in line with the Common European Framework of Reference for Languages (Council of Europe, 2001). Production and interaction proficiency are assessed through the NGT Functional Assessment instrument, adapted from the Sign Language Proficiency Interview (Caccamise & Samar, 2009). With this test we were able to determine that after one year of NGT-study students produce NGT at CEFR-level A2, after two years they sign at level B1, and after four years they are proficient in NGT on CEFR-level B2. As a result of that we were able to identify NGT texts that were matched to the level of students at certain stages in their studies with a CEFR-level. These texts were then analysed for sign familiarity, morpheme-sign rate, use of space and use of non-manual signals. All of these elements appear to be relevant for the determination of a good alignment between the difficulty of NGT signed texts and the targeted CEFR level, although only the morpheme-sign rate appears to be a decisive indicator.

TOEIC®/LPI RELATIONSHIPS IN ACADEMIC AND EMPLOYMENT CONTEXTS IN THAILAND

ABSTRACTThe Test of English for International Communication (TOEIC®) is an ETS‐developed measure of proficiency in English as a second language (ESL) or foreign language (EFL), designed to assess aspects of English language listening comprehension (LC) and reading comprehension (RC). In general samples of native speakers of Japanese, Korean, French, Spanish, and Arabic, respectively, scores on the TOEIC have been found to be strongly related to performance on the Language Proficiency Interview (LPI) procedure, a widely recognized direct measure of speaking proficiency. In this exploratory study, TOEIC/LPI relationships in subgroups (e.g., by gender, educational level, organizational setting at time of assessment) were assessed within a general national/linguistic sample—native speakers of Thai who are involved in operational assessments under the aegis of the TOEIC representative agency in Thailand, the Center for Professional Assessment. Observed relationships were relatively strong and positive in the total sample and within each of the subgroups studied. Observed coefficients were noticeably larger in subgroups of females than in corresponding subgroups of males and for a sample of university students than for examinees tested in places of employment. Means of observed LPI rating for the respective subgroups were consistent with expectations based on TOEIC scores. Further research is needed to assess the extent to which the subgroup findings may tend to be generalizable in the Center (Thailand) context or elsewhere.

Investigating accommodation in language proficiency interviews using a new measure of lexical diversity

Lexical diversity is an important indicator of language learners’ active vocabulary and how it is deployed. Traditionally it has been measured by the Type-Token Ratio (TTR), the ratio of different words to total words used. Unfortunately, TTR is a function of sample size: larger samples of words will give a lower TTR and even commonly used measures derived from TTR which are claimed to be independent of sample size are problematic. To overcome this, the authors have developed an innovative measure of vocabulary diversity, D, based on mathematically modelling how new words are introduced into larger and larger language samples, and have produced software (vocd) to calculate it. Previous research by the authors into language proficiency interviews (Richards and Malvern, 2000) investigated linguistic and discourse accommodation of teacher-testers using a wide range of student and teacher variables. In a study of teenage learners of French, the aspect of teachers’ language in oral interviews that was most responsive to the ability of their students was lexical diversity. The analysis reported here focuses on this finding in greater depth using the new measure, D. The relationship between D and other measures of foreign language proficiency is investigated, the Ds of students and teachers are compared and the correlations between teachers’ D and students’ proficiency are computed. Results firstly demonstrate the validity of D as a measure of vocabulary diversity and the effectiveness of vocd as a tool to analyse language data. Secondly, with regard to accommodation processes in oral testing, the two teachers did not finely tune their vocabulary diversity to the proficiency of individual students. Instead, each teacher roughly adjusted his or her language to the ability of the class they examined.

OVERESTIMATION OF LPI RATINGS FOR NATIVE‐KOREAN SPEAKERS IN THE TOEIC TESTING CONTEXT: SEARCH FOR EXPLANATION

ABSTRACTRegression‐based guidelines have been developed for predicting ratings of speaking proficiency in English as a foreign language (EFL), as assessed using the Language Proficiency Interview (LPI) procedure, from scores on the TOEIC® (Test of English for International Communication) Test. These guidelines, which reflect the regression of LPI rating on TOEIC scores in a combined sample composed of native speakers of Japanese, French, Spanish, and Arabic, respectively, were found to overestimate LPI rating when applied to data for native speakers of Korean. It was hypothesized that this outcome would be expected if development of EFL speaking proficiency (as assessed by the LPI procedure) tends to lag relatively more behind development of the proficiencies that are assessed by the TOEIC Test, for native‐Korean‐speaking EFL learners than for demographically comparable EFL learners in other national/linguistic settings (at least those represented in the study sample). An indirect assessment of this hypothesis was undertaken in a series of exploratory analyses involving section‐ and total‐score means for native‐language groups on the familiar, three‐part version of the Test of English as a Foreign Language™ (TOEFL®). Findings were interpreted as being generally, albeit indirectly, supportive of the developmental lag hypothesis, as well as of working assumptions underlying introduction and analysis of TOEFL native‐language reference group means as group analytic variables. Needed lines of inquiry are suggested.

TALKING AND TESTING: DISCOURSE APPROACHES TO THE ASSESSMENT OF ORAL PROFICIENCY.Richard Young and Agnes Weiyun He (Eds.). Amsterdam: Benjamins, 1998. Pp. x + 395. NLG 138 cloth.

The goal of this volume is to provide a rich, interdisciplinary foundation of empirical studies describing the discourse of Language Proficiency Interviews (LPIs), focusing on “interactional competence.” The studies not only show how difficult it is to validly assess oral proficiency; they also show that discourse analysis of LPIs—as the editors point out—is very much in its infancy. Most of these studies face research design challenges (e.g., small sample size, sample comparability, coding reliability, or a combination of these), yet they illustrate the potential contributions of various (mostly qualitative) discourse analytic approaches to an understanding and assessment of oral proficiency.

VALIDITY OF GLOBAL SELF‐RATINGS OF ESL SPEAKING PROFICIENCY BASED ON AN FSI/ILR‐REFERENCED SCALE

ABSTRACTThis study explored the validity and potential utility for self‐assessment of speaking proficiency in English as a second language (ESL), of a self‐rating scale intended to nominally parallel–in terms of number of points or levels and corresponding descriptions of language‐use behavior–a scale used by members of the Interagency Language Roundtable (ILR) to rate performance in Language Proficiency Interviews (LPI). The study sample was composed primarily of educated, adult ESL users/learners, most of whom were native‐speakers of either French or German, who provided self‐assessments just prior to taking the Test of English for International Communication (TOEIC), in employment‐related ESL training contexts in Switzerland. Correlations between self‐ratings and TOEIC scores centered around .7, and the mean ILR‐referenced self‐rating was very close to that for “predicted ILR‐scaled LPI rating”, where prediction was based on a previously developed equation for predicting LPI rating from TOEIC score. Lines of inquiry warranting further research are suggested.

Conversational Styles in Language Proficiency Interviews

This study compares the conversational styles of intermediate and advanced learners of ESL in language proficiency interviews. Eleven intermediate learners and 12 advanced learners participated in a regular administration of the Cambridge First Certificate in English oral interview. I analyzed interview discourse constructed by both interviewer and nonnative speakers (NNSs), using a quantitative model of topical organization. I found differences in the amount of talk and rate of speaking (advanced learners talked more and faster than intermediate learners), in the extent of context dependence (advanced learners elaborated more in answers to questions), and in the ability to construct and sustain narratives (advanced learners did so, intermediate learners did not). There were no differences between the two groups in the frequency of initiation of new topics, nor in the reactivity to topics introduced by the interviewers. Interviewers did not vary in their interviewing style with the two groups. Some of these findings contradict what designers of language proficiency interviews claim to be proficiency‐related differences between NNSs. The discrepancies may be due to the format of the interview and/or to differing expectations of the interview by interviewers and NNSs from different cultures.

Relationships of English grammar and communicative fluency to reading in deaf adolescents

Abstract We studied two groups of congenitally deaf students 16 to 18 years of age: one group consisted of deaf children of deaf parents and the second group consisted of deaf children of hearing parents. There were 65 subjects in each group. We studied the relationships of the dependent variable of reading with three measures of communicative fluency and two measures of English grammar/structure. For both groups there were high correlations between reading and the two measures of English grammar/structure; the Test of Syntactic Abilities and the Signed English Morphology Tests. Correlations ranged from .55 to .70, indicating that both tests are high predictive of reading for deaf adolescents. Moderate (.30 and .42), although statistically significant, correlations were found between reading and oral and manual language proficiency interviews for both groups. Fluency in American Sign Language (ASL), as measured by an ASL language proficiency interview, was not correlated with reading for either group. The...

ENHANCING THE INTERPRETATION OF A NORM‐REFERENCED SECOND‐LANGUAGE TEST THROUGH CRITERION‐REFERENCING: A RESEARCH ASSESSMENT OF EXPERIENCE IN THE TOEIC TESTING CONTEXT

ABSTRACTThis study was undertaken to develop guidelines for making interpretive inferences from scores on the Test of English for International Communication (TOEIC), a norm‐referenced test of English‐language listening comprehension (LC) and reading (R) skills, about level of ability to use English in face‐to‐face conversation, indexed by performance in the Language Proficiency Interview (LPI) situation. LPI performance, rated according to behaviorally defined levels on the LPI/ILR/FSI quasi‐absolute proficiency scale, was treated as a context‐independent criterion, using the familiar regression model in an apparently novel application (for such criterion‐referencing purposes) in the context of a large‐scale ESL‐testing program. The study employed TOEIC/LPI data‐sets generated during operational ESL assessments in representative TOEIC‐use settings (places of work or work‐related ESL training) in Japan, France, Mexico, and Saudia Arabia, involving samples of adult, educated ESL users/learners in or preparing for ESL‐essential positions with companies engaged in international commerce. The pattern of TOEIC/LPI concurrent correlations was consistent across samples and there was relatively close fit between sample LPI means and estimates from TOEIC scores, especially TOEIC‐LC, using combined‐sample regression equations. Theoretical and pragmatic implications of the findings are discussed. General guidelines are provided for making inferences about LPI‐assessed level of oral English proficiency from TOEIC scores. Directions are suggested for further research and development activities in the TOEIC testing context.

Adaptation of the Language Proficiency Interview (LPI) for Assessing Sign Communicative Competence

The LPI, a test for assessing communicative competence in a foreign or second language has been adapted for use as a Sign Communication Proficiency Interview (SCPI). 1 Discussion covers (a) an SCPI Rating Scale, (b) linguistic and cultural factors important for sign communicative competence, and (c) skills in communicating simultaneously in signing and speaking.