Oral Proficiency Test Research Articles

Incoherent coherence? Using systemic functional linguistics to improve oral language assessment literacy

AbstractThis paper discusses the challenges of defining coherence in the context of oral language assessment literacy and proposes that better understanding of the construct can be achieved through a systemic-functional linguistic lens. Coherence is taken to be a foundational quality of written and spoken discourse and is a standard feature in the assessment rubrics of most large-scale English-language oral proficiency tests such as IELTS, TOEFL, and Cambridge. However, the precise meaning of coherence, and how it can be identified in language, is difficult to grasp. This has implications for the makers and raters of tests as well as the candidates who take them. The literature suggests coherence is the most troublesome category to rate reliably and validly. The systemic-functional linguistic (SFL) model of language provides a rich description of coherence, but has been under-utilised by the language testing industry to date. In this paper we examine three aspects of coherence: (1) how it is currently conceptualised in oral language proficiency testing; (2) how it is conceptualised in linguistics in general and SFL in particular; and (3) how insights from SFL could inform the evaluation of coherence in oral language proficiency examinations.

Exploring the Grammatical Complexity of International Teaching Assistants: A Comparative Corpus Study

AbstractThe purpose of this study is to analyze the grammatical complexity features of international teaching assistants' (ITAs) mock‐teaching presentations and to compare the distributions of these features to those found in the Oral English Proficiency Test (a local ITA assessment), university classroom teaching, conversation, and academic writing. The data consisted of 186 prospective ITAs' mock‐teaching presentations collected from two ITA training courses at a large U.S. university: one higher‐level course and one lower‐level course. All presentations were transcribed, proofread, and built into a corpus consisting of 247,043 words. Based on the previous corpus studies, a total of 22 grammatical complexity features were selected for the analysis. Overall, the discourse of ITA mock‐teaching involved a substantial number of both the grammatical complexity features commonly found in spoken conversation (e.g., finite adverbial clauses and modals) and those commonly used in academic writing (e.g., noun premodifiers and prepositional phrases). The frequency distributions of the features differed in complex ways between ITA mock‐teaching and the other registers, as well as between the higher‐level and lower‐level ITAs. Pedagogical implications for ITA training and assessment, and future research directions are discussed at the end.

Is Second Language Attrition Inevitable After Instruction Ends? An Exploratory Longitudinal Study of Advanced Instructed Second Language Users

AbstractMost second language acquisition (SLA) research has documented the processes involved in learning second/foreign languages, with few studies focusing on the durability of attained second language (L2) skills once instructed learners/users are no longer receiving formal instruction. The current study examines the effects of continued exposure and peak instructional attainment on the long‐term evolution of advanced, instructed L2 learners’ skills following a longitudinal mixed‐methods research design. Participants (n = 28) completed an oral proficiency test, an oral interview, and a vocabulary knowledge test at multiple times over an 8‐year period, 6 years of which were postinstruction. Results showed that continued exposure contributes to long‐term retention (and some further development) of oral proficiency and fluency and that peak attainment at the end of formal instruction is also an important variable for some areas of L2 performance. Additionally, even the participants with limited exposure demonstrated little attrition over time.

Exploring the relationships between ASS indices and CAF and the impact on Chinese college students’ oral English performance

Automatic Speech Scoring (ASS) has increasingly become a useful tool in oral proficiency testing for Second Language (L2) learners. However, limited studies investigate the alignment of ASS indices with the Complexity, Accuracy, and Fluency (CAF)—the three dimensions in evaluating L2 speakers’ oral proficiency, and the subsequent impact indices on the oral performance of Chinese college students. To bridge this gap, this study used comparative analysis, Pearson analysis, and linear regression analysis to delve into the relationship and correlations between paired ASS indicators of “pronunciation”, “fluency”, “integrity”, “speed”, “duration”, and “overall”, while also analyzing the relationships between “overall” and other variables. These analyses were conducted using 956 audio clips of freshmen who took the College English Test-Spoken English Test Band 4 (CET-SET-4) in May 2022 in China. The findings reveal that (1) the ASS indicators and evaluation methods are similar but not identical to those employed in prior studies; (2) “pronunciation” encapsulates both the accuracy and fluency dimensions of CAF; (3) “pronunciation” and “integrity” have significant impacts on Chinese college students’ oral English performance in read-aloud tasks. The study suggests that future research should further investigate the specific pronunciation challenges faced by Chinese college students, such as phonetics, stress, and intonation. Additionally, it highlights the need to comprehend teachers’ attitudes and preferences towards ASS to enhance its effectiveness in assessing second language (L2) learners’ oral proficiency. The study would provide some references to teachers for oral English teaching design and students for their self-assessment of oral English proficiencies.

Venäläisen aksentin tunnistaminen suomen suullisen kielitaidon arvioinnissa

The paper focuses on the Russian accent in the Finnish oral proficiency test. The data consist of spontaneous monologues on an intermediate proficiency level from 10 speakers (about 90 s each). Raters (n=44) were asked to identify the accent in the speech samples as well as to give justifications for their recognition in their own words. The analysis is based on a classification of the raters’ comments as well as auditory and acoustic analysis by the author. The results show that the raters paid most attention to individual sounds, but also palatalization and intonation were mentioned as the reasons for detection of the Russian accent. To conclude, further and more detailed acoustic analysis is needed to gain more understanding of the phonetic features of Russian accented Finnish.

The Influence of Learning Strategy on the English Oral Proficiency of English Majors under the Background of Media Convergence

With the gradual development of information and media technologies, more and more tools are available in the process of language learning. In this context, the impact of the use of learning strategies on the spoken English proficiency of English majors has received widespread attention. In this study, the use of metacognitive, cognitive, and social-emotional strategies on the learning strategies of oral English was studied through questionnaires and oral English proficiency tests. The results show that the frequency of use of different learning strategies is significantly positively correlated with the oral English level of English majors in the context of media integration, that is, the use of learning strategies has a positive effect on the improvement of English-speaking level of English majors.

Prompt effects on assessment of interactional competence in a group oral test

AbstractThis study serves as an attempt to validate an in‐house oral proficiency test (hereafter referred to as OPT) with a particular focus on prompt effects on assessment of interactional competence, an issue that has attracted burgeoning interest but not been adequately investigated yet. Prompts employed in the parallel versions of the group discussion task, which were supposed to be comparable, were found to differ in two dimensions, that is, topical domain and syntactic structure. The transcribed interactions from 136 sophomore test takers (34 groups) sitting in the first administration of the OPT were coded through a rigorous procedure and examined using conversation analysis. Five patterns of interaction were identified with distinct features based on two dimensions, mutuality and equality. Marked differences in distribution of patterns of interaction were detected in discourse elicited by different prompts, which should be mainly attributed to the syntactic structure of the prompts. The open‐question prompts were found more likely to elicit “conversation‐like” interaction with higher mutuality, characterized by the collaborative and unbalanced‐collaborative patterns, while the closed‐question prompts tended to elicit more “solo versus solo” interaction with lower mutuality, featured by the parallel and unbalanced‐parallel patterns. The influence of topic domain, however, was minimal. The results thereby constitute rebuttals that weaken meaningfulness and impartiality of the claim of interpretations about the construct within Bachman and Palmer's assessment use argument framework and suggest close attention from the test developers. The implications for prompt design, and development and validation of the group oral test are further provided.

Effects Of Online Peer Learning On Speaking Performance And Language Learning Anxiety

This study investigated the effects of an online peer learning program on undergraduate student English speaking proficiency, language anxiety, and perceptions of online peer learning. The study adopted a case study design and recruited 32 undergraduate students from the Department of Applied Foreign Languages at a university in central Taiwan. The online peer learning program improved students’ English communication skills and reduced students’ learning anxiety by facilitating interactions with online foreign peers. Quantitative data from oral proficiency tests and the Foreign Language Classroom Anxiety Scale were collected before and after the online peer learning program. Qualitative data were analyzed and coded to determine students’ perceptions of the online peer learning program. A significant improvement in the students’ oral proficiency was observed in the pretest and posttest scores. However, no significant reduction in overall speaking anxiety levels was noted, although the students reported a noticeable decrease in anxiety levels following the intervention, particularly when they talked about general or familiar topics. The results of this study demonstrate that the online peer learning program effectively enhanced the speaking proficiency of students of English as a foreign language and moderately alleviated their learning anxiety. On the basis of these findings, this study provides suggestions for foreign language teachers.

Assessing First-Cycle Primary School English Language Teachers’ Speaking Proficiency

This study aimed to assess the speaking proficiency of English teachers in the first-cycle primary school employed a convergent mixed design method, with a specific focus on the East Wollega Zone. Utilizing sophisticated probability sampling, fifteen and ten English teachers underwent randomly assigned Oral Proficiency Tests and classroom observations, respectively. Descriptive statistical analysis uncovered a notable deficiency in overall speaking proficiency, as indicated by a combined mean score of 2.65, falling below the required standard. Independent sample t-tests revealed subpar Oral Proficiency Test outcomes for both male and female teachers, with p-values exceeding .05 for all components except pronunciation. Through a comprehensive assessment integrating quantitative and qualitative approaches, this study consistently highlighted below-average speaking proficiency among English teachers. The results emphasize the imperative for targeted interventions and professional development initiatives to elevate the speaking proficiency of primary-level English teachers. The research provides critical insights by identifying specific areas of deficiency in speaking proficiency, particularly in pronunciation. These findings underscore the intricate nature of language proficiency and serve as a foundational guide for future research and policy considerations to enhance English language instruction in the early stages of education, addressing identified proficiency gaps and fostering a more effective learning environment.

The Effectiveness of Using Inside Outside Circle Technique towards Students’ Speaking Ability at Tenth Grade Student of Senior High School 8 Muaro Jambi

This study aims to investigate the effectiveness of using Inside Outside Circle Technique Towards Speaking Skill at Eleventh Grade student of Senior High School 8 Muaro Jambi. The research methodology used in this study was quantitative research design with true experimental research. The population of the research was eleventh grade student and the sample was XI IPA 2 as experimental class and XI IPA 3 as control class. The instrument of the research was the oral proficiency test. The data were gathering in this research through pre-test, treatment and post-test. The data were analyzing in this research through t-test formula. As expected, the current study, it was found that the coefficient of t-(observation) = 4,05 where the coefficient of t-(table) = 1,967. It means that there is a significant effect of using Inside Outside Circle technique toward students English speaking skill at eleventh grade student of senior high school 8 Muaro Jambi.

Relationship Between Language Anxiety and English Speaking Performance

This study investigated the relationship between language language anxiety and English speaking performance among Taiwanese college students. The participants were 59 students attending a one-semester course entitled English Speaking Communication at a university in central Taiwan. They underwent computer-based oral proficiency tests and completed the Foreign Language Classroom Anxiety Scale. Data were analyzed using descriptive statistics and Pearson product-moment correlation. The English majors experienced moderate levels of anxiety when taking the computer-based speaking test, and a negative but nonsignificant correlation was discovered between foreign language anxiety and speaking performance on the computer-based exam. Pedagogical suggestions for second-language educators are presented herein. 

Oral English Proficiency Tests, Interpretive Labor, and the Neoliberal University

Tests of English proficiency for international graduate students at US universities are neoliberal institutions which make (mis)communication the responsibility of individual workers. While cloaking themselves in a discourse of linguistic expertise, they require test‐takers to assimilate to white, upper class, American mannerisms. In this ethnographic study of two testing centers, we address their material and ideological consequences: increases in precarity and xenophobia, losses in pay and students' communicative competence. We trace tests' distribution of interpretive labor (Graeber 2015) and propose a new interactionally informed approach to intelligibility which accounts for the co‐operation (Goodwin 2018) of multiple subjects.

Effects of intelligent personal assistants on EFL learners’ oral proficiency outside the classroom

Intelligent personal assistants (IPAs) are a valuable tool in language learning because they provide opportunities for authentic interaction. However, their effectiveness, compared with that of human interlocutors, in facilitating second and foreign language interaction has not been explored. Therefore, this study investigated the effect of IPAs (i.e. Google Assistant on smartphones) on English as a foreign language (EFL) learners’ oral proficiency outside the classroom and the differences between IPA-human interaction and human-only interaction. A total of 89 college freshmen participated in an out-of-class program consisting of 10-minute sessions twice a week for one semester. The participants were randomly divided into three groups: (1) those who interacted with Google Assistant on smartphones; (2) those who interacted with L1 English speakers; and (3) those who interacted with L2 English speakers. Both quantitative (English oral proficiency tests) and qualitative data (focus group interviews) were collected and analyzed. The results revealed that the out-of-class use of Google Assistant significantly improved the EFL learners’ oral proficiency, with a positive effect similar to that of interaction with L1 English speakers. A detailed analysis of the interviews revealed that the mobility and ubiquity of Google Assistant exposed learners to a large amount of high-quality oral input, provided opportunities to practice speaking with immediate, multimodal feedback, engaged learners through various modes of interaction, eliminated learners’ fear of making mistakes, reduced learners’ anxiety in speaking English, and encouraged self-directed learning outside the classroom, which are all conducive to improving EFL learners’ oral proficiency.

The Implementation of Collaborative Dialogues in a Literary-Cultural Course

Several researchers (e.g., Allen & Paesani, 2010; Maxim, 2009; MLA Report, 2007) argue that the language-literature divide limits language development in many foreign language departments and that the speaking skill is the most affected by this common two-tiered curriculum (Swender, 2003). This study investigates the implementation of the concept of collaborative dialogues in an upper-division Francophone literature and culture course to support the oral proficiency skills of the participants. It addresses research questions pertaining to how they constructed their group conversations in terms of language and content. Both whole-class discussions and weekly group dialogues, which took place outside of class, were video-recorded. The participants took an oral proficiency test at the beginning and at the end of the study and shared their opinions about the dialogues in two questionnaires. The analysis of the data sources shows that the majority of participants focused heavily on content during their conversations. This finding differs from previous research on collaborative dialogues, which fostered many interactions about language and supported language learning. Based on their analytical abilities and proficiency levels, the participants of this study either reviewed previous class discussions or extended them by exploring additional material and adding prior knowledge to their arguments.

Using narratives of personal experience in L2 oral language proficiency testing

Using narratives of personal experience in L2 oral language proficiency testing

Policy in Practice: Teachers’ Conceptualizations of L2 English Oral Proficiency as Operationalized in High-Stakes Test Assessment

Assessment of foreign/second language (L2) oral proficiency is known to be complex and influenced by the local context. In Sweden, extensive assessment guidelines for the National English Speaking Test (NEST) are offered to teachers, who act as raters of their own students’ performances on this high-stakes L2 English oral proficiency (OP) test. Despite guidelines, teachers commonly construct their own NEST scoring rubric. The present study aims to unveil teachers-as-raters’ conceptualizations, as these emerge from the self-made scoring rubrics, and possible transformations of policy. Data consist of 20 teacher-generated scoring rubrics used for assessing NEST (years 6 and 9). Rubrics were collected via personal networks and online teacher membership groups. Employing content analysis, data were analysed qualitatively to examine (i) what OP sub-skills were in focus for assessment, (ii) how sub-skills were conceptualized, and (iii) scoring rubric design. Results showed that the content and design of rubrics were heavily influenced by the official assessment guidelines, which led to broad consensus about what to assess—but not about how to assess. Lack of consensus was particularly salient for interactive skills. Analysis of policy transformations revealed that teachers’ self-made templates, in fact, lead to an analytic rather than a holistic assessment practice.

Codeswitching of bilingual preschoolers in Pakistan: A study of Arabic and Urdu alphabets

Urdu is the official language of Pakistan, while Arabic is the language of the Holy Quran, Muslims' sacred book. Pakistan, as a Muslim country with a religious bond, makes it obligatory for Muslims to recognize and interpret Allah's commandments, as well as to pray and recite the Holy Quran. The most effective skill is oral communication. It has become one of the fundamental language learning skills, alongside listening, reading, and writing. Pronunciation is the process of processing speech sounds to communicate. The current research study focuses on the codeswitching of bilingual preschoolers while learning the Arabic and Urdu alphabets. The study demonstrates why preschool children fail to recognize images of simple words in Arabic and Urdu at the same time and how this challenge can be tackled with the ingenuity of a committed teacher. The researcher used the Oral Proficiency Test (OPT) to assess the quality of the children's pronunciation, and he processed the data using Setiyadi’s (2020) Triangulation of method, which includes three data collection techniques: observation, interview, and questionnaire. The observation focused on how the teacher pronounced both alphabets and how the children responded by pronouncing the same at the top of their voices to make good use of articulators. Later, an interview and a questionnaire were conducted to collect data that could be used to ensure the validity of the observational results. Creswell & Baez’s (2020) Model of the Structure of Study was implemented. The research on bilingual preschoolers is significant in that it reveals that the children had difficulty in discerning resembling sounds from the Arabic and Urdu alphabets and that this difficulty caused them to deviate from one alphabet to the other. Speech therapists, pathologists, psychologists, bilingual scholars, and researchers will all benefit from the research.

Investigating and optimizing score dependability of a local ITA speaking test across language groups: A generalizability theory approach

With the present study I investigated the sources of score variance and dependability in a local oral English proficiency test for potential international teaching assistants (ITAs) across four first language (L1) groups, and suggested alternative test designs. Using generalizability theory, I examined the relative importance of L1s (i.e., Indian, Korean, Mandarin, and Spanish), examinees, tasks, and ratings to score variability, and estimated dependability across the L1s. The analyses identified examinees as the largest contributor, which is important for high dependability and validity arguments for test scores. Effects of ratings and tasks were small, but L1 effects on score variance were considerable, with the Indian group’s dependability lowest. Unlike previous generalizability theory studies on L1 effects, however, further analyses revealed that the L1 effects highly likely reflect proficiency differences rather than strong bias when comparing the percent agreement of the ratings, external criteria of examinee English proficiency, and underlying score distributions. I discuss the proficiency differences related to varied socio-linguistic contexts of using and learning English. Lastly, I suggest an alternative design with fewer items and one additional rating for improved dependability. Considering multiple test purposes specific to ITA testing (i.e., efficiency, construct representation, formative advantages), I propose a flexible approach.

Developing and validating a computerized oral proficiency test of English as a foreign language (Coptefl)

This study is about the development and validation process of the Computerized Oral Proficiency Test of English as a Foreign Language (COPTEFL). The test aims at assessing the speaking proficiency levels of students in Anadolu University School of Foreign Languages (AUSFL). For this purpose, three monologic tasks were developed based on the Global Scale of English (GSE, 2015) level descriptors. After the development of the tasks, it was aimed to develop the COPTEFL system and then compare the test scores and test-takers’ perspectives on monologic tasks between the COPTEFL and the face-to-face speaking test. The findings from these quantitative and qualitative analyses provided substantial support for the validity and reliability of the COPTEFL and inform the further refinement of the test tasks.

Correlates of the Oral Language Performance of Sixth Grade Students Exposed to Multilingual Instruction in Private Schools in Tacloban City

Multilingual instruction is a common practice nowadays wherein various languages are used as medium of instruction. This descriptive study sought to determine the oral language performance of grade six students enrolled in trilingual instruction in some private schools in Tacloban City, Philippines. Primarily, it aimed to find out the profile of the students in terms of some student, home and teacher factors; determine whether there is significant correlation between these factors; determine if they correlate with oral language performance; and, identify the most important predictor that may influence oral language performance. A total of 37 respondents from the 115 sample population drawn through simple random sampling, the respondents were asked to answer questionnaires and were given oral proficiency tests in English, Filipino and Chinese. The results revealed that the students were labelled as Limited Speakers in English and Non-Speakers in both Filipino and Chinese. Their oral language performance was influenced by the students’ attitude towards Filipino, their motivational orientation, the kind of school enrolled in, as well as the teachers’ age and years of teaching experience. The kind of school enrolled in found to be a significant predictor of oral language performance of the students.