English Speaking Test Research Articles

Improving elementary EFL speaking skills with generative AI chatbots: Exploring individual and paired interactions

Generative artificial intelligence (GAI) and automatic speech recognition (ASR) have ushered in promising tools for foreign language learning, notably GAI chatbots. This study investigated the impact of GAI chatbots on elementary school English as a foreign language (EFL) learners' speaking skills, focusing on two interaction configurations—individual and paired. Eighty-five elementary school EFL learners participated in a three-week summer program, engaging in daily 45-min interactions with CoolE Bot. The participants were randomly assigned to three groups: (1) individual interaction with CoolE Bot (I-Bot group), (2) paired interaction with CoolE Bot (P-Bot group), and (3) interaction with teachers and peers in a conventional English classroom (No-Bot group). In each class, participants in the Bot group received worksheets with a topic, prompts, and vocabulary to guide their interactions with CoolE Bot, while those in the No-Bot group also received worksheets for comparable activities. Quantitative (English-speaking tests) and qualitative data (semi-structured interviews) were collected and analyzed. Results revealed that the I-Bot and P-Bot groups' post-test speaking skills were significantly higher than those of the No-Bot group. CoolE Bot significantly improved the speaking skills of EFL learners. Both individual and paired interactions with CoolE Bot demonstrated positive effects, with no significant differences between groups. Interviews highlight CoolE Bot's adeptness in coherent interaction, charismatic conversational style with a human-like voice, diverse topic discussions tailored to learners' interests, and supportive functions. The participants found GAI chatbot-assisted EFL speaking enjoyable, motivating, and engaging appreciating its cartoonish, human-like characters, conversational style, and voice. Additionally, CoolE Bot fostered rapport and a supportive environment enhancing learners' confidence and reducing anxiety regarding EFL speaking. Individual interactions encourage personalized engagement and self-directed learning, whereas paired interactions involve social dynamics, shared learning experiences, and mutual resolution of language challenges.

Test review: Computer-based English Listening and Speaking Test (CELST) of National Matriculation English Test (NMET) Guangdong version in China

Test review: Computer-based English Listening and Speaking Test (CELST) of National Matriculation English Test (NMET) Guangdong version in China

The Effect of Using the Contemplative Approach to Enhance Speaking and Critical Thinking Skills in the EFL Undergraduate Literature Classroom

This research study explored the effect of using the contemplative approach to enhance speaking skills and critical thinking skills in the EFL literature classroom which aimed to 1) investigate English speaking skills and 2) investigate students’ critical thinking skills in the EFL literature classroom. This research study applied a mixed-method research design. Data collection was conducted in both qualitative and quantitative formats. Instruments were an English speaking test, a critical thinking test, classroom observations, semi-structured interviews, and a reflective journal. The sample group was eleven second-year students of the English studies major who enrolled in the literary adaptation course in the second semester of the academic year 2022 at Chiang Rai Rajabhat University, Thailand by purposive sampling with the volunteers who were interviewed and joined the activities. The quantitative data was analyzed by descriptive statistics while the qualitative data was classified, analyzed, and reported by the objective items. The findings indicate that contemplative approaches can significantly enhance students' English-speaking skills. Although the results suggest that participating in contemplative activities did not have a statistically significant impact on the participants' critical thinking abilities, a slight improvement was observed in their critical thinking skills.

The analysis of marking reliability through the approach of gauge repeatability and reproducibility (GR&R) study: a case of English-speaking test

Ensuring consistent and reliable scoring is paramount in education, especially in performance-based assessments. This study delves into the critical issue of marking consistency, focusing on speaking proficiency tests in English language learning, which often face greater reliability challenges. While existing literature has explored various methods for assessing marking reliability, this study is the first of its kind to introduce an alternative statistical tool, namely the gauge repeatability and reproducibility (GR&R) approach, to the educational context. The study encompasses both intra- and inter-rater reliabilities, with additional validation using the intraclass correlation coefficient (ICC). Using a case study approach involving three examiners evaluating 30 recordings of a speaking proficiency test, the GR&R method demonstrates its effectiveness in detecting reliability issues over the ICC approach. Furthermore, this research identifies key factors influencing scoring inconsistencies, including group performance estimation, work presentation order, rubric complexity and clarity, the student’s chosen topic, accent familiarity, and recording quality. Importantly, it not only pinpoints these root causes but also suggests practical solutions, thereby enhancing the precision of the measurement system. The GR&R method can offer significant contributions to stakeholders in language proficiency assessment, including educational institutions, test developers and policymakers. It is also applicable to other cases of performance-based assessments. By addressing reliability issues, this study provides insights to enhance the fairness and accuracy of subjective judgements, ultimately benefiting overall performance comparisons and decision making.

Adieu Bias: Debiasing Intuitions Among French Speakers.

Recent debiasing studies have shown that a short, plain-English explanation of the correct solution strategy can improve reasoning performance. However, these studies have predominantly focused on English-speaking populations, who were tested with problem contents designed for an English-speaking test environment. Here we explore whether the key findings of previous debiasing studies can be extended to native French speakers living in continental Europe (France). We ran a training session with a battery of three reasoning tasks (i.e., base-rate neglect, conjunction fallacy, and bat-and-ball) on 147 native French speakers. We used a two-response paradigm in which participants first gave an initial intuitive response, under time pressure and cognitive load, and then gave a final response after deliberation. Results showed a clear training effect, as early as the initial (intuitive) stage. Immediately after training, most participants solved the problems correctly, without the need for a deliberation process. The findings confirm that the intuitive debiasing training effect extends to native French speakers.

Developing and Testing Audio Data Processing Modules in Python to Connect to and Data Be Scored by ASS Cloud Server

Automatic Speech Scoring (ASS) system developed on a basis of automatic speech recognition (ASR) technology is a powerful computer-assistant tool for oral test scoring. However, due to the limits of high equipment costs and high-tech operating costs of a local ASS, ASS cloud services have become the first choice of most oral English teachers and learners. The purpose of this paper is to develop and test modules in Python to preprocess the audio data, connect to the cloud server, and convert JSON data format into common Excel form. 1056 pieces of audio data were collected from test-takers’ read-aloud task of CEST-4 (College English Speaking Test band 4)) and six variables (i.e., “pronunciation”, “fluency”, “integrity”, “speed”, “duration”, and “overall”) were defined. After analyzing the data of the test results, it is found that the oral test score is mostly affected by the “pronunciation” and “integrity”, and the accuracy of pronunciation is the strongest predictor of oral performance. The modules and functions are helpful for teachers and students to use in daily oral test/practice, and these modules can also be employed in other second language oral test scored by ASS cloud sever, like oral Chinese test. Our results can provide reference and guidance for future oral research and teaching.

Construction of an innovative model of English online teaching based on a multimedia English learning system model

Abstract This paper explores a multimedia learning system suitable for students’ independent English learning through the current development of assisted English learning systems, university-level English teaching requirements, and learner characteristics. Based on the model of the proposed multimedia English learning system, in the endpoint detection stage of speech, short-time excess rate and short-time energy are combined to process the endpoint detection of speech. In the final speech comparison stage, the dynamic time planning algorithm DTW is analyzed and improved before the similarity recognition of speech. We also complete the requirement analysis of the English-speaking learning system by combining the relevant theories and applying the speech signal processing technology and template matching technology to realize a multimedia-assisted learning system suitable for college students to conduct independent English learning. Through the English-speaking test of two classes, the control class using traditional online teaching had many failures reaching 25%, while the experimental class using the English-assisted learning platform passed all of them. 80 points or more were achieved by only 5% of the control class, while 20% of the experimental class. The average score of the control class was 63.4, while the experimental class was 72.1, and the experimental class was 8.7 points higher than the control class. Therefore, teaching English-assisted learning platforms can improve students’ English-speaking abilities.

How sociocultural factors mediate washback effect: a study of the Hong Kong Diploma of Secondary Education Examination English Language Paper 4

The study explores the unaddressed washback on the English-speaking test of the Hong Kong Diploma of Secondary Education Examination (HKDSE-English Language Paper 4). The HKDSE is a university entrance exam in Hong Kong which includes a group discussion speaking test known as English Language Paper 4. To identify ongoing washback effects, including sociocultural perspectives, the study analyses ongoing student washback of test-specific and non-specific preparation and how the mediating factors affect these preparations. One-hundred and ninety-nine secondary 4th to 6th grade students provided the questionnaire data in two rounds. Results from exploratory factor analysis indicate six major types of test preparation, which are categorised as two test specific, three non-specific, and one integrated type of preparation. Mediating factors comprise five categories of learner characteristics and stakeholders in and out of school. The cluster analyses identified four students’ groups intertwined with the involvement of washback and the mediating factors. A strong relationship between extrinsic factors and explicit learning and intrinsic factors and implicit learning was found. The study concludes that students’ washback is affected by mediating factors aside from the test; learning method choice by students (e.g. learning from entertainment content) has a strong relationship with specific mediating factors (e.g. interest in English).

A Critique of the Computer-Based English Speaking Test in Fujian (CEST-FJ)

This essay applies Bachman and Palmer’s (1996) Test Usefulness Framework to scrutinize the reliability, construct validity, and impact of the Computer-based English Speaking Test in Fujian (CEST-FJ). The CEST-FJ’s inclusion in the National College Entrance Examination (NCEE) represents a positive step towards promoting communicative ability in English language learning in China. The computerized administration and scoring processes enhance the test’s reliability and control over construct-irrelevant variance. However, limitations such as potential variations in difficulty, construct under-representation of communicative competence, challenges for pronunciation assessment in the context of English as an International Language (EIL), and the non-compulsory and low-value nature of the test have been identified. Additionally, the lack of transparency surrounding the CEST-FJ hampers assessment and accountability. This essay emphasizes the need for more theoretical and empirical research to justify or enhance the test in the future, particularly in light of its high-stakes nature and influence on test-takers’ university admissions. Test designers are urged to prioritize transparency and provide more information to facilitate further research and ensure the effectiveness of the CEST-FJ.

Enhancing critical thinking and English-speaking skills of gifted students through philosophy for children approach

This study aimed at investigating the effectiveness of "Philosophy for Children (P4C)" approach on the critical thinking and English-speaking skills of twenty-three 7th grade gifted students learning English as a foreign language at a Science and Art Center (SAC) in the city of Denizli, Türkiye. In the study a mixed methods research design was adopted with the participation of randomly assigned experimental and control groups. While quantitative data were collected through the Cornell Critical Thinking Test (CCTT) X level and the Cambridge English Speaking Test (A2 level for Schools), qualitative data were collected through reflective diaries written by the experimental group participants and evaluation forms. While the control group followed the usual SAC English speaking lessons, English speaking lessons for the experimental group were based on the P4C approach. According to the quantitative data findings, there was an increase in the scores of both groups in terms of speaking skills in English, however this difference was not statistically significant. In terms of critical-thinking skills, there was a slight decrease between the pre-test and post-test scores of the control group and an increase in the scores of the experimental group and this difference was statistically significant. In addition, the qualitative data revealed that the experimental group participants generally provided positive feedback on P4C practices and speaking lessons based on the P4C approach created positive effects on their critical thinking and speaking skills in English, while such an application did not create a significant effect on their speaking skills in English.

Developing Dataset Management of Speech Recognition Based Automatic English-Speaking Skill Testing System

Speech recognition technology has been widely used to aid English language testing, such as for pronunciation and speaking practice for various use cases including job interview simulation. We developed an app, powered by speech recognition technology and regex text processing, that automatically score participants’ speaking performance. However, this application lacks dataset management for the speaking test maker to manage the indicators of accepted answers. A feature to effectively manage the dataset and efficiently ease the speaking test creation is needed. This paper presents the development of dataset management feature to support the speech recognition based automatic English-speaking skill testing system. All elements that support the feature in development were tested through Blackbox testing with various scenarios. The elements were proven to work as intended to support the existing system. The new feature has brought efficiency for the English-speaking test maker.

Speaking Test Apps Development: English Learning Enhancement in Online Learning Complexity

Abstract. This research tried to map the strength and weaknesses and evaluate the English teaching and learning processes in Balinese EFL learners during the online class. The use of IT in online learning has become a central issue for the decreasing of students’ motivation due to the lack of students – teachers’ interaction in the online class especially in English subject. The complexities in the online learning include how the use of IT can be synchronized with teaching and learning methods in delivering the learning material, how is the role of the teacher in motivating students, how to assess students' abilities in online learning, and many more. Those are the unavoidable obstacles faced by teachers and students during the online learning. This study was aimed at evaluating and making English online teaching and learning effective and creating IT-based application to optimized the teaching and learning process and the students’ performance. This qualitative research collects the data through observation , interviews and English speaking test to 15 students and six teachers. From the empirical studies of the Balinese EFL learners, the students stated that English learning in online classroom atmosphere were ineffective and unsupportive for their English skill development. The problems mainly arise from the lack of practice and interaction which caused a decrease in student motivation and interest in online learning. To be able to overcome those problems, the researcher created learning applications called Speaking Test App. and Teacher Speaking Test App. to facilitate teachers and students in learning process. This application can be freely accessed through the Playstore and Appstore. The students and teachers use the learning applications as interactive English learning mediums in the classrooms. The results of the study showed that with the technology-based learning media, the students' performance, enthusiasm and motivation in learning could be improved and highlight a significant impact for interaction in learning especially in English online learning.Keywords: English teaching and learning, IT-based learning, Online Learning

Investigating a shared-dialect effect between raters and candidates in English speaking tests.

This study set out to examine existence of a shared-dialect effect, a phenomenon that when a rater shares the same dialect with a candidate, the rater is more likely to give the candidate a higher score in English speaking tests. Ten Cantonese-speaking raters and ten Mandarin-speaking raters were selected to assess forty Cantonese-accented and forty Mandarin-accented candidates' oral performance in the retelling task of the Computer-based English Listening and Speaking Test (CELST). Besides, seven raters from each group participated in the stimulated recall stage aiming to reveal their thought process. Quantitative results suggested that the two rater groups were comparable in terms of internal consistency. There were no significant differences in the scores of both candidate groups awarded by both rater groups. The effect of interaction between candidates' dialect and raters' dialect was not statistically significant, indicating non-existence of such effect. Qualitative results showed that some raters attended to candidates' accents, and indicated that awareness of accents and their familiarity with the accents affected their comprehension of the speech samples and potentially influenced their scoring process. The findings are discussed with reference to rater training, rating scale, raters' familiarity with candidates' accents, raters' attitudes toward candidates' accents and the task type. The main implication of this study is that recruiting both group raters in domestic English speaking tests is warranted if the shared-dialect effect could be duly managed.

IMPROVING ENGLISH STUDENTS’ SPEAKING SKILLS THROUGH AN ACTION LEARNING STRATEGY

For students of English education, speaking skills are of paramount importance. However, in the context of English students at Universitas Bengkulu, many students seemed to still struggle with speaking skills. Therefore, to cope with this problem, this classroom action research was conducted with the aim of improving the speaking skills of students who took English conversation course by applying action learning strategy (ALS) since this strategy was scientifically promising. Two phases of the class action design were employed in this research project. The data were garnered using observations, interviews, and English-speaking test. It was evident from the first phase of observation that almost all students recommended the ALS. The data from the interview revealed that a significant proportion of students had profited from this approach, and there were few students who spoke in front of the class. The first phase's precise decision how an average score of 74 was attained by the students. At the second phase, the observation and interview data showed that nearly all students actively participated during discussions and had good improvements of English-speaking skills. The result revealed that the implementation of ALS in the conversation course in semester 1 improves S1 students’ speaking skills.

The Delivery of Speaking Tests in Traditional or Online Proctored Mode: A Comparability Study

This paper investigates the comparability of test scores recorded for high-stakes English language Speaking Tests administered face-to-face in either a traditional centre-based mode (TM) or in an online proctored mode (OLP). The data comprise a large sample of test takers taking English language Speaking Tests at four CEFR (the 'Common European Framework of Reference for Languages') levels – B1 to C2 – via TM or OLP. The data were analysed using descriptive statistics, effect size differences and equivalence tests. While a degree of difference in scores obtained between modes was apparent at C2 level, the differences were not found to be statistically significant. The paper concludes that whether Speaking Tests are delivered in online proctored mode or in traditional face-to-face mode, test takers receive similar scores. The study confirms that mode of test delivery does not significantly affect test taker scores.

Panel Discussions: Skilled Communication And Knowledge Construction to Develop EFL Learners’ Speaking Performance

The poor speaking performance is a big obstacle for EFL learners to communicate and to construct knowledge. The main aim of the study was to find out how panel discussion can be implemented to develop EFL learners’ speaking performance. In panel discussion, students need to understand the topic and to construct knowledge by gathering information and deeply engaged in learning. The study followed action research methodology where data was gathered from an in-depth interview and Test of Spoken English (TSE), collected from 18 students in Universitas Muhammadiyah Jember. The speaking score was better than before the implementation of panel discussion, as seen from the mean score in TSE (64.63 in Preliminary to 70.93 in Cycle 1). The highest achievement made is in the aspects of vocabulary (67% better). Panel discussion can significantly improve speaking because the process of knowledge construction stimulates students to have extended and multimodal communication.

A Study on the Design of English Speaking Examination System Based on SSM Framework

At present, the teaching, training, and assessment programs of English speaking courses in schools are basically carried out by the teaching method of face-to-face teaching by teachers, especially in the English speaking assessment; students and teachers communicate with each other to answer questions, while the students’ performance is subjectively judged by the school English teachers. This model not only consumes the teaching resources of school English teachers but also cannot ensure the uniformity of grading standards. To this end, this paper designs and proposes an English speaking test system with the aim of building an intelligent speaking test system by designing a speech recognition deepening frequency processing function through SSM framework combined with JSP technology and deep learning theory. The English speaking test system designed in this paper integrates several open source frameworks, adopts modular design, and divides it into two subsystems, student-side module system and teacher-side module system for system framework design, and finally, this system is tested for functionality. The system was designed and implemented through the framework description, functional description, and functional testing of this system, and finally, the English speaking test system was designed and implemented. The system is designed and implemented to change the traditional one-to-one mode of school teaching, save teachers’ manpower, strengthen the content of intensive training for students’ English proficiency and improve the frequency of exams, facilitate school administrators and English teachers to evaluate the effectiveness of their teaching implementation, and develop corresponding effective individualized teaching and guidance strategies for schools to provide convenience, which can greatly reduce the recognized labor intensity of English teachers and provide a correct evaluation of students, a correct evaluation of the level, and other roles.

Implementing Open Educational Resource to Strengthen English Speaking Skills

The purpose of this study was to analyze the improvement of English speaking skills from the application of Open Educational Resource (OER). The research method uses Exploratory Mixed Method. Qualitative data was taken during the observation process, while quantitative data was taken from the results of the pre-test and post-test. The participants were 4th-semester students who took the Speaking 2 course. The number of students who participated was 60 people. Qualitative data were analyzed through the data reduction stage, then the data presentation, and the conclusion stage. Meanwhile, the data on the results of the practice and the English-speaking test were presented in the form of descriptive statistics and t-tests. The results showed that OER had a positive impact on improving English speaking skills. This can be understood from the diversity of OER types used during the learning process. The results of the study have implications for the understanding of educators related to the provision of learning resources that are to the needs of students and current technological developments so that learning activities are relevant to the changes and needs of 21st-century educational competencies.

Self-Study System Evaluation of Oral English Based on Limit Theory

With the globalization of the contemporary social economy and its ever-increasing influence, it is particularly important to master a common language in the world at this stage English. For the subjects who have the opportunity to learn English, their listening, reading, writing, and other abilities are relatively good, but the speaking ability is relatively lacking. Therefore, the self-study of spoken English is particularly important for the improvement of the practical application ability of English, and an efficient self-study system will also play a better role in the self-study process. The efficiency of a system depends on the maximum and minimum magnitudes that the system can carry. Therefore, for the self-learning system of spoken English in this paper, it is necessary to calculate the maximum and minimum carrying capacity. Therefore, it is necessary to deal with the limit thinking of the maximum and minimum carrying capacity of the system. The evaluation based on limit theory in this paper is a kind of evaluation of the maximum and minimum carrying capacity of the English oral self-learning system by using the convergence and dispersion characteristics of limit thinking. In order to evaluate the performance of the oral English self-learning system, this paper implements the dynamic evaluator of the oral English self-learning system, and constructs a hardware system for the self-learning oral English. This paper also uses the hidden Markov model in the self-learning system and the application of the central limit theory. The corresponding experimental results show that the mean value of the spoken English test data is 25.1337, and the standard deviation is 2.01385, which is in line with the normal distribution. It shows that the evaluation based on the limit theory can make the operation of the spoken English self-learning system stable.

Lessons learned from five years of speaking exam administration

From 2015 to 2019, the Bunkyo English Communication Center at Hiroshima Bunkyo University conducted end-of-semester speaking exams called Bunkyo English Speaking Tests (BESTs) for all English Communication freshman and sophomore students. During these five years, the Bunkyo English Communication Center learned several test administration best practices. First, in a desire to apply a many-facet Rasch model using the Facets software package (Linacre, 2022a) to provide student fair scores that account for rater leniency and severity, a preventative flaw in the rater schedule was discovered and corrected. Second, the increased complexity of the rater schedule plus a desire to streamline the exam processes necessitated the building of a comprehensive scheduling and testing system in Excel. Finally, the calculation method initially used for converting Rasch measures into student fair scores was based on a faulty assumption and suffered from ambiguity and subjectivity, and a fairer workaround system was discovered and implemented. This paper documents the discovery of these problems and the process of developing and implementing their solutions.