Oral Proficiency Test Research Articles

Japanese Language Skill of Korea Japan Children - With the Focus of Connection to Oral Proficiency Test in a Qualitative Assessment

Japanese Language Skill of Korea Japan Children - With the Focus of Connection to Oral Proficiency Test in a Qualitative Assessment

The Investigation of Learning Strategies of American Learners of Chinese and Japanese for Character Learning

It has been widely recognized that Chinese and Japanese languages are exceptionally difficult to learn. One of the reasons is their logographic characters (i.e. hanzi in Chinese, kanji in Japanese) that are extremely different from alphabet-based orthography (Tong & Yip, 2015; Xu & Padilla, 2013). Accordingly, there have been research investigating how L2 learners of Chinese and Japanese deal with the difficulty by exploring learners’ strategy (Gamage, 2003; Shen, 2005). However, learning strategies for a certain aspect of characters (i.e. shape, sound) have not been investigated as much as learning strategies in general (but see Shen, 2005). In addition, there are limited longitudinal research exploring how learners change their strategies. Therefore, the researchers investigate strategies that L2 learners of American university students are using most frequently for Chinese and Japanese character learning. The study had 66 L2 learners taking either Chinese or Japanese course at an American university. They took a questionnaire at the beginning and at the end of a semester. It was found that reading, context, decomposition, rote-writing, and listening were the most frequently used strategies. Moreover, the results indicated that strategies vary depending on which aspect of characters they learn. Furthermore, learners did not change their learning strategies over three months to a notable extent.
 References
 
 Chikamatsu, N. (1996). The effects of L1 orthography on L2 word recognition: A study ofAmerican and Chinese learners of Japanese. Studies in Second Language Acquisition, 18,403–432.
 Everson, M. E. (1998). Word recognition among learners of Chinese as a foreign language:Investigating the relationship between naming and knowing. The Modern LanguageJournal, 82, 194–204.
 Everson, M. E. (2011). Best practices in teaching logographic and non-Roman writingsystems to L2 learners. Annual Review of Applied Linguistics, 31, 249–274.
 Haththotuwa Gamage, G. (2003). Perceptions of kanji learning strategies: Do they differamong Chinese character and alphabetic background learners?
 Hayes, E. B. (1988). Encoding strategies used by native and non‐native readers ofChinese Mandarin. The Modern Language Journal, 72, 188–195.
 Ke, C. (1998). Effects of language background on the learning of Chinese charactersamong foreign language students. Foreign Language Annals, 31, 91–102.
 Liskin-Gasparro, J. (1982). ETS Oral Proficiency Testing Manual. Educational TestingService, Princeton, NJ.
 McGinnis, S. (1999). Student goals and approaches. Mapping the course of the Chineselanguage field, 151–188.
 Mori, Y., Sato, K., & Shimizu, H. (2007). Japanese language students' perceptions on kanjilearning and their relationship to novel kanji word learning ability. LanguageLearning, 57, 57–85.
 Packard, J. L. (1990). Effects of time lag in the introduction of characters into the Chineselanguage curriculum. The Modern Language Journal, 74, 167–175.
 Rose, H. (2013). L2 learners' attitudes toward, and use of, mnemonic strategies whenlearning Japanese kanji. The Modern Language Journal, 97, 981–992.
 Shen, H. H. (2005). An investigation of Chinese-character learning strategies among nonnative speakers of Chinese. System, 33, 49–68.
 Tong, X., & Yip, J. H. Y. (2015). Cracking the Chinese character: radical sensitivity inlearners of Chinese as a foreign language and its relationship to Chinese wordreading. Reading and Writing, 28, 159–181.
 Yuki, M. (2009). Kanji Learning Strategies: From the Viewpoint of Learners with Nonkanji Background. 関西外国語大学留学生別科日本語教育論集, 19, 143–150.
 Xu, Y., Chang, L. Y., & Perfetti, C. A. (2014). The Effect of Radical‐Based Grouping inCharacter Learning in Chinese as a Foreign Language. The Modern Language Journal, 98,773–793.
 Xu, X., & Padilla, A. M. (2013). Using meaningful interpretation and chunking to enhancememory: The case of Chinese character learning. Foreign Language Annals, 46, 402–422.
 
 

Equity in L2 English oral assessment: Criterion-based facts or works of fiction?

For assessment to be equitable, it is central that teachers/raters perceive and apply grade criteria similarly. However, in assessing L2 oral proficiency in paired tests, raters must grade test-takers individually on a joint interaction performance. With a conversation analytic approach, we examine closely one recording from a 9th-grade national test of L2 English with an aim to uncover some aspects that underpin vastly divergent assessments (as assigned by three raters) of one test-taker. Findings pointing to issues such as moral stance, rater experiences, and the interlocutor effect are discussed in light of equity in L2 oral proficiency testing and assessment.

English

Lexical Fluency Units (LFUs) are strings of words that act like single items in language users’ mental lexicon. Recently, there has been a resurgence of interest in analyzing them in learners’ speech and classifying them into different types (Ushigusa 2008, Wood 2012). The literature in this particular field of study stated that the higher use of lexical fluency units might lead to the higher score on tests of oral English proficiency; therefore, the present study investigated the relationship between lexical fluency, temporal fluency (TF), and proficiency. To conduct the study, forty Iranian EFL teachers with TOEFL scores above 570 were chosen. To ensure that they have acquired competency in communication by which they can be employed as an instructor in the real context, they took an Oral English Proficiency Test (OEPT). Then, the sounds recorded from OEPT were analysed by PRRAT and Transcriber software and the transcripts of their responses were also analysed for detection of lexical fluency units (LFUs). Results indicated that teachers who incorporated more LFUs in their speech had higher TF ratio and consequently higher proficiency score. The interpretations and implementations are also discussed. Keywords: lexical fluency; temporal fluency (TF); lexical fluency units (LFUs); language proficiency; lexicon DOI: http://doi.org/10.17576/3L-2016-2201-09

Testing L2 Talk: A Review of Empirical Studies on Second‐Language Oral Proficiency Testing

AbstractIn this review article, empirical studies published from 2004 through 2014 on second‐language (L2) oral proficiency testing are analyzed, with a specific focus on discourse and social interaction in such tests. Taking three common test setups, oral proficiency interviews (OPIs), paired peer tests, and group peer tests as an organizing principle for the studies examined, recent developments in L2 oral proficiency testing research are situated, reviewed, and discussed, with a particular focus on tests conducted in face‐to‐face contexts. Findings from the review of selected journals, databases, monographs, and edited collections indicate (1) a prevalence of studies of the OPI format, but a growing research base on paired and group tests, (2) an absence of oral test studies in discourse journals, and (3) an emphasis on assessment, validation, and rater perspectives, as opposed to detailed analyses of interaction in L2 tests.

Turn-taking practices in multi-party EFL oral proficiency tests

This study focuses on multi-party talk among EFL learners during peer-to-peer interaction in oral proficiency tests. We conduct a micro-analysis of the way test-takers in small groups use questions like ‘How about you?’ to manage turn-taking when speaking in rounds or through a pivot. We find that these deceptively simple questions have both an indexical referential element and an addressee-determining element that is used in conjunction with bodily conduct to select next-speaker, making them in fact a sophisticated interactional achievement. The study concludes by discussing how turn-taking practices in multi-party speaking tests can help make visible novice language users’ orientations to learning.

Quantitative Analyses of Test-takers' Speech Samples in a Semi-direct Oral Proficiency Test: Focus on Proficiency Difference.

Quantitative Analyses of Test-takers' Speech Samples in a Semi-direct Oral Proficiency Test: Focus on Proficiency Difference.

Towards a framework for assessing foreign language oral proficiency in a large-scale test setting

This paper presents the qualitative findings of a study that applied Dynamic Assessment (DA) to an English as a foreign language (EFL) oral proficiency test within a large-scale context in secondary schools in outlying areas in Israel. Two formats of mediation were scrutinized. Micro-genetic analysis of learners’ discourse during their interactions present several salient features showing rich potential for application in a large-scale setting: knowledge construction, self/peer interaction, self-criticism, use of L1 to advance L2, blurred roles of participants in a community of practice, group-DA and group ZPD. Clearer understanding of DA mechanisms and the current findings may lead to greater and more effective use of dynamic assessment in oral proficiency testing, especially with large groups.

Research on the Constructs of Oral Proficiency Tests in Korean: a case of discourse production tasks

Research on the Constructs of Oral Proficiency Tests in Korean: a case of discourse production tasks

An examination of rater performance on a local oral English proficiency test: A mixed-methods approach

This paper reports on a mixed-methods approach to evaluate rater performance on a local oral English proficiency test. Three types of reliability estimates were reported to examine rater performance from different perspectives. Quantitative results were also triangulated with qualitative rater comments to arrive at a more representative picture of rater performance and to inform rater training. Specifically, both quantitative (6338 valid rating scores) and qualitative data (506 sets of rater comments) were analyzed with respect to rater consistency, rater consensus, rater severity, rater interaction, and raters’ use of rating scale. While raters achieved overall satisfactory inter-rater reliability ( r = .73), they differed in severity and achieved relatively low exact score agreement. Disagreement of rating scores was largely explained by two significant main effects: (1) examinees’ oral English proficiency level, that is, raters tend to agree more on higher score levels than on lower score levels; (2) raters’ differential severity due to raters’ varied perceptions of speech intelligibility toward Indian and low-proficient Chinese examinees. However, effect sizes of raters’ differential severity effect on overall rater agreement were rather small, suggesting that varied perceptions among trained raters of second language (L2) intelligibility, though possible, are not likely to have a large impact on the overall evaluation of oral English proficiency. In contrast, at the lower score levels, examinees’ varied language proficiency profiles generated difficulty for rater alignment. Rater disagreement at these levels accounted for most of the overall rater disagreement and thus should be focused on during rater training. Implication of this study is that interpretation of rater performance should not just focus on identifying interactions between raters’ and examinees’ linguistic background but also examine the impact of rater interactions across examinees’ language proficiency levels. Findings of this study also indicate effectiveness of triangulating different sources of data on rater performance using a mixed-methods approach, especially in local testing contexts.

The Influence of Partner Acquaintanceship in an EFL Speaking Test

This paper explores the effect of the familiarity between partners on their performance during oral proficiency tests in an EFL class. Not many studies about familiarity between test-takers and their partners have been conducted, but such sociocognitive factor appears to be empirical with regard to performance tests(O'Sullivan, 2002; Porter, 1991). Along the line of research conducted by O'Sullivan(2002) and Porter(1991), this paper investigates the role of interlocutors with respect to their familiarity with test-takers. Japanese and Korean learners of English participated in two speaking tests; paired with their friends in the first speaking test and paired with partners who they are not familiar with in the second speaking test. Under the two different familiarity conditions, the test-takers participated in two different types of English oral proficiency tests: 1) a task focusing on grammar(sentence formation) and 2) a task measuring their overall L2 proficiency(impromptu speech). The findings of the research are as follows. First, the overall results show that the test scores are positively correlated with familiarity. Second, the impact of familiarity is greater on the complex L2 tasks such as impromptu speech than the simple tasks such as sentence-forming. Third, Japanese participants exhibit more individual difference whereas Korean participants are more dominantly affected(positively) by being grouped with the familiar partners. (University of Aizu․Busan University of Foriegn Studies)

The Effects of Type and Quantity of Input on Iranian EFL Learners’ Oral Language Proficiency

In the written texts on foreign language learning, a group of studies has stressed the function of learning context and learning chances for learners’ language input. The present thesis had two main goals: on the one hand, different types of input to which Iranian grade four high school EFL learners’ are exposed were looked at; on the other hand, the possible relationship between types and quantity of input and Iranian EFL learners’ oral proficiency was investigated. It was supposed that EFL learners who have access to more input will show better oral proficiency than those who do not have. Instruments used in the present study for the purpose of data collation included PET test, researcher- made questionnaire, oral language proficiency test and face- to -face interview. Data were gathered from 50 Iranian female grade four high school foreign language learners who were selected from among 120 students whose score on PET test were + 1SD from the mean score. The results of the Spearman rank –order correlation test for the types of input and oral language proficiency scores, showed that the participants’ oral proficiency score significantly correlated with the intended four sources of input including spoken (rho= 0.416, sig=0.003), written (rho= 0.364, sig=0.009), aural (rho= 0.343, sig=0.015) and visual or audio-visual types of input (rho= 0.47, sig=0.00). The findings of Spearman rank –order correlation test for the quantity of input and oral language proficiency scores also showed a significant relationship between quantity of input and oral language proficiency (rho= 0.543, sig= 0.00). The findings showed that EFL learners’ oral proficiency is significantly correlated with efficient and effective input. The findings may also suggest answers to the question why most Iranian English learners fail to speak English fluently, which might be due to lack of effective input. This may emphasize the importance of the types and quantity of input in EFL classes and syllabus designers.

The Effect of Using Drama on Improving Preparatory Year Students' Oral Proficiency at Al-Imam Muhammad Ibn Saud Islamic University

This study aims at investigating the effect of using drama as a teaching technique on improving the oral proficiency of the preparatory year students in the applied sciences stream at Al-Imam Muhammad Bin Saud Islamic university. The sample of the study consists of 42 male preparatory year students distributed in two sections, one constituted the experimental group, and the other constituted the control group. A pre-post test was used to measure the oral proficiency of the two groups. The experiment took place during the first semester of the academic year 2011/ 2012. In order to answer the question of the study, the researchers used the Oral Interview test as an eliciting tool. The interview was scored according to an oral assessment criteria designed by the researcher, based on the Foreign Service Institute Oral Interview (FSI) (Oller, 1979). After going through the dramatization and improvisation techniques, the two groups were post tested using the same oral proficiency test. The means of the pre and post tests for the two groups were calculated. A t-test was applied to analyze the results of the subjects on the pretest and post-test at the level of significance ( statistical significant differences in the oral proficiency favoring the experimental group who was taught according to dramatization and improvisation techniques based on dialogue formats. In the light of these findings, the researchers presented some conclusions and recommendations.

Such a Nice Gesture: Paired Spanish Interaction in Oral Test Discourse

The problem faced by researchers regarding the construct of 'interaction' in paired peer-peer tasks in oral language proficiency testing is addressed by analysing verbal protocols of raters watching candidates in a videoed paired speaking task in a test in Spanish as a foreign language context. The aim is to investigate any correspondence between observable nonverbal features in the candidates' videoed speech sample that raters claim to attend to while observing paired interaction and those features of body language that candidates are aware of while performing the test. This study is different to previous studies because the task is for paired beginner foreign language students of Spanish. The data is drawn from 17 transcribed paired speech samples, transcribed Verbal Protocols collected from 12 Spanish teacher/raters and 25 candidates Stimulated Verbal Recalls. The analysis is guided by non verbal communication performance features already identified in two previous studies: a previous rater cognition study where raters comment on their observation of candidate performance through Verbal Protocols (Ducasse and Brown, 2009) and a candidate Stimulated Verbal Recall where candidates comment on their interaction in a test performance (Ducasse, 2007). Findings indicate that of the non verbal communication features gaze and gesture are equally important to candidates during performance and to raters during rating process. These findings support findings in Conversation Analysis and Gesture studies and have implications for rating peer-peer language proficiency and for the 'interaction' construct.

Diverging task orientations in L2 oral proficiency tests – a conversation analytic approach to participant understandings of pre-set discussion tasks

Abstract Presumably most students strive to do well in school and on national tests. However, even in standardized tests, students’ and examiners’ expectations on what it means to ‘do well’ may diverge in ways that are consequential to performance and assessment. In this paper, we examine how students and teachers in an L2 English peer–peer speaking national test (9th grade) display their understandings of appropriate ways of dealing with pre-set discussion tasks. Using conversation analysis and 38 recorded national tests in English in Sweden, we demonstrate, e.g., how teachers’ displayed understandings of how tasks should be appropriately handled steer the interactional trajectory between students in particular directions. The analysis shows that participants spend much time on negotiating understandings of the task-at-hand. We argue that in terms of valid assessment of oral proficiency, task understandings merit more attention, as task negotiations inevitably generate different conditions for different dyads and teachers.

A semi-structured English oral proficiency test for certification of teachers in South Africa

South Africa has eleven official languages, but English is by far the most commonly used medium of instruction in schools. It is therefore essential that the English proficiency of preservice teachers be accurately assessed by means of a valid, reliable and feasible assessment instrument. As there are so many tertiary institutions responsible for teacher certification in the country, standards of oral assessment vary widely. This paper describes a semi-structured oral proficiency test, designed to meet these needs. Possibilities for further development of the test are explored Suid-Afrika erken elf amptelike tale, maar Engels is die medium wat verreweg die meeste in skole gebruik word Dit is dus noodsaaklik dat onderwysstudente se taalvaardigheid in Engels akkuraat gemeet moet kan word dm. v. 'n geldige, betroubare en toepaslike meetinstrument. Die feit dat so baie tersiere inrigtings in Suid-Afrika betrokke is by die sertifisering van onderwysers, lei onvermydelik tot groot skommelinge in standaarde. Hierdie artikel beskryf 'n semigestruktureerde mondelinge taalvaardigheidstoets wat ontwikkel is om in die behoeftes te voorsien. Moontlikhede vir die verdere ontwikkeling van die toets word ondersoek.

Test-Takers’ Strategic Behaviors in Independent and Integrated Speaking Tasks

This study investigated the strategic behaviors that test-takers reported using when responding to integrated and independent speaking tasks in an English oral proficiency test [the Speaking Section of the Internet-based Test of English as a Foreign Language™ (TOEFL iBT)] and the relationship between test-takers’ strategic behaviors and their test scores. Each of 30 Chinese-speaking engineering students responded to two independent and four integrated speaking tasks and provided stimulated recalls about the strategies they used when performing each task. The integrated tasks elicited a wider variety of reported strategy use than did the independent tasks. Additionally, the integrated tasks were more alike with respect to reported strategy use than were the independent and integrated tasks. Overall, we found no relationship between the total number of reported strategic behaviors and total test scores, regardless of task type. Our finding that the more skills involved in a task, the greater the reported strategy use supports the inclusion of integrated tasks in oral proficiency tests. However, the relationships between strategy use, task type, and task performance are varied and complex.

The contribution of test-takers’ speech content to scores on an English oral proficiency test

The content that test-takers attempt to convey is not always included in the construct definition of general English oral proficiency tests, although some English-for-academic-purposes (EAP) speaking tests and most writing tests tend to place great emphasis on the evaluation of the content or ideas in the performance. This study investigated the relative contribution of linguistic criteria and the elaboration of speech content to scores on a test of speaking proficiency. A speaking test was designed and administered to Japanese undergraduates to determine what criteria English teachers associate with general oral proficiency. Nine raters were recruited to rate 30 students’ monologues on three topics, using intuitive judgments of oral proficiency (referred to as Overall communicative effectiveness). Following this, they assigned scores to the monologues using five criteria: Grammatical accuracy, Fluency, Vocabulary range, Pronunciation, and Content elaboration/development. The raters were also asked to provide open-ended written comments on the factors contributing to their intuitive judgments. Statistical analyses of the scores – Rasch measurement, multiple regression, and multivariate generalizability (G) theory analysis – revealed that Content elaboration/development made a substantive contribution to the intuitive judgments and composite score. The present study enriches our understanding of general oral proficiency and the construct definition of proficiency tests.

<b>Testing oral proficiency: what does pronunciation tell us?</b><br>DOI:10.5007/2175-8026.2011n60p247

In this study we discuss the role of pronunciation in languagetesting and investigate two features of pronunciation of eightcandidates of the Test of Oral Proficiency in English (TEPOLI)along four bands of the test scale. Deviations in vowel segmentsand in syllable structure are analyzed. The results point to theneed for a global assessment of the candidates’ phonologicalsystems.

The Effect of Mode of Response on a Semidirect Test of Oral Proficiency

This article reports on a study conducted with 42 participants from a Chilean university, which aimed to determine the effect of mode of response on test performance and test-taker perception of test features by comparing a semidirect online version and a direct face-to-face version of a speaking test. Candidate performances on both test versions were double-marked and analysed using both classical test theory and many-facet Rasch measurement. To gain an insight into students' perceptions of the two modes of delivery, we also asked candidates to complete a questionnaire after sitting each version. The many-facet Rash analysis showed no significant difference in the difficulty of the two versions of test. Nonetheless, there was a significant preference among candidates for the face-to-face version across a number of different features of the test.