English Language Testing Research Articles

Be a Realistic Walker During the Dark Time: A Mirror Dream

Jianqiao Chen’s journey into an English-language post-secondary institution began with the International English Language Testing System, otherwise known as the IELTS exam. In this story, written for WRS 101 class, Jianqiao takes the readers on his personal quest as he prepares to take the exam. Part realistic struggles, part dream, the paper shows how hard work, self-reflection and regular practice yield positive results.

Exploring the interplay of digital storytelling, L2 speaking skills, self-regulation, and anxiety in an IELTS preparation course

This investigation explores the potential of integrating digital storytelling (DST) within an English as a Foreign Language (EFL) course specifically designed for International English Language Testing System (IELTS) preparation. The study employs a randomized controlled trial methodology, involving 89 EFL learners divided into two groups. The primary objective is to assess the effectiveness of DST intervention on three key variables: L2 speaking skills, speaking self-regulation, and speaking anxiety. A pre-test, post-test, and one-month follow-up design is implemented to capture the longitudinal effects of the intervention. Data analysis utilizes Linear Mixed-Effects Models (LMM) to account for the repeated-measures nature of the study. The LMM incorporates both fixed effects (group, time, and their interaction) and random effects (participant intercepts) to provide a comprehensive evaluation of the intervention’s impact. Preliminary findings suggest that the incorporation of DST may yield positive outcomes in L2 speaking skills, speaking self-regulation, and speaking anxiety. This research aims to contribute to the existing body of knowledge by providing empirical evidence on the efficacy of DST as a pedagogical tool within IELTS preparation contexts. The findings hold potential value for language educators and practitioners seeking to enhance speaking proficiency and foster a more positive learning environment for learners experiencing anxiety.

An automatized semantic analysis of two large-scale listening tests: A corpus-based study

This study examined the semantic features of the simulated mini-lectures in the listening sections of the International English Language Testing System (IELTS) and the Test of English as a Foreign Language (TOEFL) based on automatized semantic analysis to explore the content validity of the two tests. Two study corpora were utilized, the IELTS corpus with 56 mini-lectures (38,944 words) and the TOEFL corpus with 285 mini-lectures (207,296 words). The reference corpus comprised 59 lectures from the Michigan Corpus of Academic Spoken English (MICASE), totaling 571,354 words. The corpora were submitted to automatized semantic tagging using Wmatrix5. Three comparisons were conducted: IELTS versus TOEFL, IELTS versus MICASE lectures, and TOEFL versus MICASE lectures. The results suggest that IELTS and TOEFL mini-lectures shared 78% and 64% of the same semantic features as MICASE, respectively, supporting their relative content validity. Nevertheless, specific semantic categories, such as politics, war, and intimate and sexual relationships, were notably absent from the test corpora, even though they appeared in the academic lecture corpus. In addition, causal connectors are frequently used in both tests, while the mini-lectures of IELTS listening tests cover fewer academic discourse fields than TOEFL mini-lectures. Implications for content validity are discussed.

Factors Affect In IELTS Test For Students of STIKOM Uyelindo Kupang

IELTS (International English Language Testing System) is a standardized English language proficiency test which is a prerequisite for continuing study or work. There are still many people who experience difficulties in carrying out the IELTS test to achieve a score above 6. The aim of the research is to know what are the factors that affect the result of students IELTS test. This research wants to find out what factors influence students' mastery of IELTS material. The method used in this research is descriptive qualitative. Qualitative descriptive research uses a series of methods to specify and describe data findings that occur in the field without providing experimental treatment. There are four IELTS materials given during the test, including: reading, writing, speaking, and listening. These four materials require strategies in their implementation and each has their own difficulties in mastering them. This research will be conducted at STIKOM Uyelindo Kupang with the research object being students of the Informatics and Information Systems Engineering Study Program Semester 2. It is hoped that this research will provide readers with an understanding regarding the success factors in carrying out the IELTS test.

Exploring the roles of ideal L2 writing self, growth L2 writing mindset, and L2 writing grit in L2 writing achievement among EFL learners

Abstract Considering the undeniable importance of examining the role of domain- and skill-specific individual difference factors in second-language (L2) writing research, this study examined the possible roles of English-as-a-foreign-language (EFL) learners’ ideal L2 writing self and growth L2 writing mindset in their L2 writing grit, which may in turn contribute to their L2 writing achievement (WA). Data were collected from 532 English-major students selected via convenience sampling by administering a questionnaire to measure their growth L2 writing mindset, ideal L2 writing self, and L2 writing grit. The International English-Language Testing System (IELTS) academic writing tasks 1 and 2 were used as measures of the learners’ L2 WA. The psychometric properties of the scales were investigated and verified through confirmatory factor analyses. Structural equation modeling (SEM) results revealed that ideal L2 writing self and L2 writing grit directly predicted L2 WA. Additionally, growth L2 writing mindset and ideal L2 writing self predicted L2 WA through the mediation of L2 writing grit. This study highlighted the importance of domain- and skill-specific individual difference factors in L2 WA and the vital necessity of considering them in L2 writing instruction.

The pain, the pedagogy, and the politics of IELTS: a critical autobiographical inquiry

ABSTRACT Language tests, including international high-stakes English proficiency tests widely used around the world, are to be viewed as ideological constructs connected with power relations and center–periphery demarcations at different social levels. In this paper, we examine the International English Language Testing System (IELTS) as an instance of such tests to uncover some of its underlying sociocultural functions. Adopting a critical autobiographical approach that revisits three personal trajectories in relation to each other and in connection with wider contextual dynamics, we reflect on our own lived experiences of engagements with IELTS in various social and institutional settings and in different roles. Each one of us first shared narrative accounts of turning points in our learning, taking, teaching, and using IELTS within the past two decades. Then, we explored these narratives through multiple rounds of reflections and discussions of their common underlying ideas related to the sociocultural functioning of IELTS. Although the narratives presented unique trajectories, three common themes of Pain, Pedagogy, and Politics emerged from our collective analytic reflections. We discuss these themes and how they can problematize different aspects of IELTS from a critical point of view.

Investigating the visual content of a commercialized academic listening test: Implications for validity

As incorporating visual modes in listening tests is gradually gaining traction in second language (L2) assessment, the inclusion of such visuals brings up questions about the role of visual modes in meaning-making during listening and test validity. In this study, we investigated the visual features of the International English Language Testing System (IELTS) listening test through the application of the social semiotic multimodal framework. Our corpus comprised 300 visuals from 256 academic listening testlets published between 1996 and 2022. Unlike the past studies of social semiotic multimodal analyses that relied on qualitative methods, our study adopted a series of visualization and quantitative statistical analysis of frequency and dispersion measures, using the general linear model to examine the visuals from a social semiotic multimodal perspective. The results revealed significant variation in the visual structures of the testlets. Through applying a post-hoc analysis, we further proposed recommendations for further research on multimodal materials in listening assessment and discussed the implications of the observed variation for the validity of the IELTS listening test. This study may be considered the first attempt to examine L2 listening assessment from a corpus-based social semiotic multimodal perspective, which may inspire more investigations on multimodal listening.

Managing Peer-Feedback Resistance In Pre-Service Language Testing And Evaluation Course

This study is situated within the English language testing and evaluation course, integral to the English language teaching curriculum, wherein senior-year preservice teachers are involved in designing language exams as a part of their evaluation criteria. They actively engage in reciprocal peer feedback sessions aimed at identifying weaknesses and problems in the exams to enhance the quality, validity, and reliability of test items, thereby cultivating an environment favorable to enhanced learning and the effective incorporation of feedback. Employing a micro-analytic approach, conversation analysis, to closely examine the interactions of preservice teachers, the study has revealed that peer groups follow a set of principles on language testing and assessment as they critically engage in delivering feedback to one another. However, recognized for its inherently problematic nature due to the emotional investment involved in feedback exchanges, interlocutors may experience high levels of anxiety and defensiveness while their work undergoes a detailed review (Pekrun et al., 2002, 2011). Accordingly, the study's main findings discovered the emergence of resistance and misalignment among peers during the feedback exchanges, leading to sequence expansion, including evaluation, justification, and recommendation until a joint consensus is achieved. Despite extensive research on resistance encounters in medical interviews, couples counseling, post-feedback sessions, and service encounters, the field remains notably deficient in studies examining peer feedback interactions in the higher education context among preservice language teachers. Therefore, this study aims to fill this gap by examining the dynamics of peer feedback interactions among preservice teachers, shedding light on how resistance to feedback unfolds among peers and is managed through the emergent phenomenon, reference to testing principles.

Admissions pathways and international undergraduates’ academic achievement

ABSTRACT Background There are a number of pathways available for international students to gain admission into universities in English-speaking countries. While many international students for whom English is not their first language succeed academically, there is concern that some do not cope with the demands of university which can affect their retention and academic success. This study considers the effects of different admissions pathways on international undergraduate students’ academic success in one New Zealand university. Purpose Findings from the first phase of this research revealed that international students who entered university with the National Certificate of Educational Achievement (NCEA) gained at high school in New Zealand performed significantly worse academically than their peers who entered through three other admissions pathways. This study aimed to ascertain whether there were salient features of students’ prior education and undergraduate experiences which contributed to this outcome. Method In-depth interviews were conducted with nine students who entered a university in New Zealand via two admissions pathways: NCEA or the International English Language Testing System (IELTS). The data generated through these interviews was analysed to enable qualitative comparisons between the two groups. Findings Results suggest that the IELTS group outperformed their NCEA peers due to maturity and prior education in their home countries. While the NCEA students had spent some years at high school in the host country, they did not, in general, demonstrate the same level of academic literacy as their IELTS peers. Conclusion International students’ outcomes appear to differ due to previous educational experiences. Students who enter via NCEA are potentially more likely to need support, as they may still be dealing with language proficiency issues, whereas those who enter via IELTS with prior higher-level education in their home country may be more academically prepared. It is important that institutions promote learning support opportunities for all international students, to ensure they experience a positive social and learning experience, which can ultimately contribute to their retention in learning, and their academic success.

Fitting the mixed Rasch model to the listening comprehension section of the IELTS: Identifying latent class differential item functioning

This study applied the Mixed Rasch Model (MRM) to the listening comprehension section of the International English Language Testing System (IELTS) to detect latent class differential item functioning (DIF) by exploring multiple profiles of second/foreign language listeners. Item responses of 462 examinees to an IELTS listening test were subjected to MRM analysis. Three classes emerged: (1) ‘Medium-level Stimulus Processors’ who can somewhat synchronize top-down and bottom-up processing, handle multitasking to a certain extent, comprehend moderately complex items, and manage input delivered at a relatively fast pace; (2) ‘High-level Stimulus Processors’ who have greater abilities in synchronizing top-down and bottom-up processing, multitasking, understanding complex items, and handling fast delivery input and more paraphrased content; and (3) ‘Low-level Stimulus Processors’ who rely more on bottom-up processing, have limited lexico-grammatical knowledge, struggle with multitasking and complex items, and find fast delivery input and paraphrased content challenging. Differences across the classes were further explained.

Cultural adaptation and psychometric analysis of Communication Activities of Daily Living third edition in Spanish and Catalan for people with aphasia.

In the Spanish and Catalan context, there is currently a lack of standardized, linguistically adapted tools to assess people with communication disorders. This lack is especially evident when it comes to instruments designed to assess functional communication. The main objective of this study is to adapt the instrument entitled Communication Activities of Daily Living 3rd edition (CADL-3) into European Spanish (CADL-3VE) and Catalan (CADL-3VC), thus providing a new tool to assess the functional communication of patients with aphasia in the Spanish and Catalan populations. A total of 152 people, all residents of Catalonia, took part in the study. The CADL-3VE test was administered to 125 Spanish-speaking participants, who were divided into two groups, one consisting of patients with aphasia and the other a control group. The CADL-3VC test was administered to 27 Catalan-speaking patients with aphasia. Other tests and assessment scales were used for the external validation of the test. Reliability scores were recorded for both new versions of the test. There was a very strong correlation between the CADL-3VE test and external criteria. The scores for both of the new versions showed significant differences in terms of performance between the aphasia and control groups. Both versions displayed similarities with the original test with respect to most of the psychometric analyses carried out. The test makes it possible to assess everyday communicative functioning and participation in real-world contexts. As such, it helps inform the creation of personalized, interdisciplinary treatment plans aimed at functional objectives that consider the patient's context. What is already known on the subject In order to effectively assess aphasia based on a bio-psycho-social perspective, it is necessary to examine functional communication. In other words, there is a need to explore the kinds of communication difficulties that people face in their everyday lives. Communication Activities of Daily Living (CADL-3) is an English-language test that was developed to assess functional communication by simulating certain everyday activities. What this study adds This paper offers an analysis of the items, reliability and validity of the Spanish and Catalan versions of the CADL-3 test. What are the clinical implications of this work? This new instrument has the potential to play an essential role in assessing the everyday functional communication of people with aphasia in Spain.

Hindi Translation, Cultural Adaptation, and Validation of Modified Weight Bias Internalization Scale (WBIS-M): A Cross-Sectional Study from the Metabolic Surgery Clinic in North India.

The lack of culturally-sensitive tool masks the epidemiology of weight-bias and internalized-stigma. This study aimed to translate, adapt, and validate the Modified Weight-Bias Internalization Scale (WBIS-M) into Hindi. The translation and adaptation process followed the World Health Organization guidelines. Three psychiatrists translated the original WBIS-M into Hindi. The bilingual panel comprised four psychiatrists, two laparoscopic bariatric surgeons, a professor of English, and an individual with severe obesity (with an International English Language Testing System (IELTS) score of Band 8.5) who was familiar with Western culture. The panel identified and resolved inadequate expressions of translations or discrepancies. An independent translator, whose mother tongue was English, back-translated it into the English language. Focused group discussions with individuals with severe obesity were conducted. The final H-WBIS-M was administered to undergraduates (n = 120) after seven days. Test-retest reliability was assessed. The original WBIS-M, Eating Disorder Diagnostic Screen, and Fat Phobia Scale-Short Form were administered to 120 undergraduates and 55 individuals with severe obesity (n = 175)]. Reliability and validity of H-WBIS-M were analyzed. Kaiser-Meyer-Olkin measure of sampling adequacy was 0.788, and Bartlett's test of sphericity was χ2 = 1381.892, p = 0.000. The exploratory factor analysis extracted three components. Spearman-Brown coefficient for H-WBIS-M was 0.875. The internal consistency was α = 0.875 (p = 0.000), intraclass correlation coefficient was 0.857 (95% confidence interval [CI] = 0.812-0.893) (p = 0.000), and Pearson correlation ranged from 0.869 to 0.989 (p = 0.000). Cross-language concordance revealed a significant intraclass correlation coefficient (0.877-0.986) (p = 0.000) and Pearson correlation ranging from 0.781 to 0.972 (p = 0.000). The H-WBIS-M demonstrated good psychometric properties based on standard testing procedures.

The predictive value of gaze behavior and mouse-clicking in testing listening proficiency: A sensor technology study

This study employed eye-tracking and mouse click frequency analysis to investigate the predictive power of gaze behaviors, mouse-clicking, and their interactive effects with linguistic backgrounds on the IELTS (International English Language Testing System) listening test scores. A total of 77 test takers (45 with English as their first language (E-L1) and 32 with English as their second language (E-L2)) participated in this study. Their eye movements and mouse click frequencies were recorded as they took a computer-based IELTS listening test. The subsequent data analysis, utilizing linear mixed models, showed that gaze patterns, mouse actions, and language background significantly predicted listening test outcomes across four listening test sections and between E-L1 and E-L2 candidates, accounting for 33.2% of the variance observed in test scores. These results indicate the effect of potential sources of construct-irrelevant variance on test scores, which are not predicted in the available construct definitions of the test used in the study. Implications for the listening construct and test validity are discussed.

Assessing student teachers’ knowledge of English to inform curriculum design in initial teacher education

Background: In Initial Teacher Education (ITE) programmes, there are concerns about student teachers’ English language proficiency.Aim: To discuss the first iteration of the PrimTEd English language and literacy test and analyse the results for information about the test instrument and about student teacher attainment.Setting: South African student teachers in BEd programmes preparing for teaching in primary schools, across six public universities, participated in the test from 2020 to 2022.Methods: The student results for the PrimTEd English assessment were analysed using basic statistics, focussing on the largest samples of first-year and fourth-year students who took both the subtests. The items in each subtest were coded according to their underlying language constructs, and each subtest was subjected to Rasch analysis before the test was considered as a whole.Results: The initial test showed the attainment of fourth-year students (mean = 55%, standard deviation [SD] = 21%, n = 731) was only 3 percentage points higher than that of first-year students (mean = 52%, SD = 20%, n = 1177).Conclusion: The similarities between the first- and fourth-year students’ test results is concerning. There remains much work to be done in building student teachers’ English language proficiency in BEd programmes.Contribution: The findings and recommendations should be useful to ITE lecturers in designing programmes that enable progressive development of English language and literacy competence from first year to graduation.

The Effect of Metacognitive Intervention on the Listening Performance and Metacognitive Awareness of High- and Low-Working Memory Capacity EFL Learners.

EFL listening comprehension has been a stark challenge for language learners, but little is known about the combined effect of individual differences, such as working memory capacity, and metacognitive intervention. Thus, the present experimental study investigates the effect of metacognitive intervention on the listening performance and metacognitive awareness of high- and low-WMC EFL learners. For this purpose, Oxford Placement Tests were distributed among 120 male Iranian EFL learners, of which 94 were identified as intermediate. Then, backward visual digit span tests were administered to measure their working memory capacity. Based on the median of all scores, 80 learners were selected and randomly assigned to two experimental groups and two control groups each with 20 participants. Next, their performance on the International English Language Testing System and the Metacognitive Awareness Listening Questionnaire was measured before and after the 8-session metacognitive intervention. Results showed that high-WMC experimental learners had a higher gain with a large effect size in terms of listening performance compared with all the other groups. In addition, the experimental group learners reported the significantly higher use of the metacognitive strategies with a moderate effect size. Interestingly, low-WMC learners' listening performance and metacognitive awareness also improved as a result of the intervention. Our findings bear pedagogical significance in that individual differences in WMC should be considered more in both EFL language classes and the future line of research involving the metacognitive intervention.

Effects of grading rubrics on EFL learners’ writing in an EMI setting

Despite considerable evidence that supports the use of grading rubrics (GRs) as tools for written corrective feedback, there is a paucity of research that investigates which of the different types of GRs best develops learners' International English Language Testing System (IELTS) writing scores in English as a medium of instruction (EMI)-contested settings. This study attempted to explore which rubric types (i.e., holistic, ESL composition profile, correction code, and IELTS) best assist English as a Foreign Language (EFL) learners in writing proficiency and which type leads to improving IELTS scores when such practice is embedded in EMI-disputed settings. Therefore, 351 male and female Saudi EFL learners were recruited to participate voluntarily. These participants were distributed equally among four groups corresponding to rubric type. For almost four months, the participants were exposed to a process-genre approach in which they were required to draft topics based on the comments received from their peer colleagues and teacher. The comments provided depended on the rubrics specified for their group type. The participants' pretest, midterm, and posttest scores were analyzed using one-way analysis of variance, t-tests, and paired samples t-tests. The results revealed that the ESL composition profile developed gradually, followed by the correction code group. However, the holistic groups did not improve. The tests were also assessed by specialists using the IELTS rubric. The findings revealed that the IELTS groups outperformed the other groups in all tests, followed by the female group in the ESL composition profile in the posttest. Meanwhile, other groups failed to improve. We discussed the results, considering the importance of GRs for improving EFL learners’ scores. Finally, we outlined the pedagogical implications for writing teachers in EMI settings. This study aimed to contribute to the growing research on EMI in relation to GRs, especially in the context of tertiary education in Saudi Arabia.

Authenticity of academic lecture passages in high-stakes tests: A temporal fluency perspective

Corpus-based studies have offered the domain definition inference for test developers. Yet, corpus-based studies on temporal fluency measures (e.g., speech rate) have been limited, especially in the context of academic lecture settings. This made it difficult for test developers to sample representative fluency features to create authentic listening passages. To address this issue, the Fluency Corpus of Academic English Lectures (FCAEL) was created to offer insight into the thresholds for temporal fluency features in academic lecture settings. The current study compared the corpus data to the academic lecture passages in the Test of English as a Foreign Language Internet-based test (TOEFL iBT) and International English Language Testing System (IELTS) to examine the domain definition inference of these tests. In total, 14 temporal fluency measures were examined. A bootstrapped one-way multivariate analysis of variance (MANOVA), followed by a series of bootstrapped analyses of variances (ANOVAs), independent t-test, and Tukey tests showed some support for the tests, although many limitations were also found. The study suggests the 25th–75th percentile of FCAEL as tentative thresholds for each temporal fluency feature. The proposal may be useful for test developers to create and revise test materials. Coding schemes, analysis codes, and raw corpus data are available on the project’s Open Science Framework page, exemplifying how Open Science can provide benefits beyond the academic community.

Listening Comprehension Skill: The Forgotten Skill in The Saudi Classrooms

Listening plays a very significant role in language acquisition and language learning. The more input kids and students get, the faster and easier it will be for them to acquire and learn any new language. Unfortunately, over a long period listening didn’t receive enough attention by curricula designers, teachers, and students which affected the output and labelled listening as a difficult skill to teach and understand. This study intends to draw the attention to listening through answering three important questions related to the importance of listening, the challenges that face teaching it and suggestions for enhancing this important skill. To this end, the researchers chose all secondary and intermediate students in Jeddah as a population and fifty male and female students as a sample. A survey of twenty- two items was used as a data collection tool. The findings of the study showed clear enthusiasm for listening comprehension and an urgent need to overcome the challenges and reconsider suggestions to improve this skill including training teachers, facilitating it, and increasing the weight in the schedules. The study recommends giving more attention to listening and including it in any English language test.

A latent profiling approach to foreign language enjoyment and L2 achievement

ABSTRACT Both Broaden-and-Build theory and Control-Value theory (CVT) posit that enjoyment contributes to academic achievement. However, empirical studies show mixed results concerning the predictive effects between these constructs in Second Language Acquisition (SLA). This study examines the link between second language (L2) achievement and Foreign Language Enjoyment (FLE) among 1,360 Chinese tertiary-level students learning English as a foreign language (EFL). Participants completed a national standardised English language test and an FLE questionnaire. Data analysis involved the use of Mplus to perform factor mixture modelling (FMM), a technique that identifies latent subgroups within the sample based on their response patterns. Results revealed four distinct profiles, each displaying different FLE-L2 achievement profile characteristics. The importance of this study lies in its analytic approach, which recognises the non-uniform connections between the three dimensions of FLE and L2 achievement across learners. This finding challenges the notion of straightforward correlations and sheds light on the complexities of these relationships, thereby addressing inconsistencies observed in previous studies and paving the way for more nuanced future research.

قياس فاعلية نموذج مقترح لتضمين مهارتي التحدث والكتابة في اختبار كفايات اللغة الإنجليزية STEP في تطويرها وفقاً لمعايير تقويم الإطار المرجعي الأوربي المشترك وفي ضوء رؤية المملكة العربية السعودية 2030 من خلال وجهة نظر المختبرين في مدينة جدة

The aim of the research is to submit a proposal to include the speaking and writing skills of the STEP English language test according to the standards of Evaluation of the Common European Framework of Reference in the of Vision 2030 Saudi Arabia's, the researcher followed the descriptive quantitative approach to achieve this goal and a list of the criteria of the Common European Framework of Reference was prepared. Speaking and writing skills, the researcher prepared a questionnaire that included these standards to ensure their availability among learners after conducting the STEP test, In addition to learners' opinions on improving their speaking and writing skills if these two skills are included in the test, The questionnaire was applied to a random sample of learners who underwent the STEP test and their number was (153) learners, Based on the researcher's findings, which were that the speaking and writing skills of the learners were not affected after the test, and the support of most learners Those who took the test in addition to speaking and writing skills and that this will contribute to improving their language and writing skills, It recommended the following recommendations: Seeking to include speaking and writing skills to test English language competencies in accordance with the evaluation of international standards, including For languages due to their flexibility and comprehensiveness, in addition to recommending teachers to seek to develop writing and speaking skills using various teaching methods Evaluate learners' participation and activities immediately to motivate learners and increase their appetite for learning English, and try to link English language learning to the reality of learners, And clarify the importance of the English language in order to achieve Vision 2030 and its importance for the integration of teachers in the labor market in addition to their integration into international universities.