English Language Proficiency Test Research Articles

From Pen to Pixel

The use of digital technologies in higher education is continually increasing, leading to changes in language use and presumably altering the language skills needed for academic studies. However, scores from high-stakes English language proficiency (ELP) tests used in postsecondary admissions only ensure the prerequisite level of traditional English skills (reading, writing, listening, and speaking). Such tests generally do not directly assess technology-mediated language skills (e.g., using online dictionaries and communicating via text message) that likely facilitate successful degree completion for international students. We present results from a needs analysis survey to re-evaluate the English-medium postsecondary linguistic landscape (i.e., update the target language use domain description), to inform ELP admissions tests. We specifically investigate international student (n = 379) and disciplinary instructor (n = 427) perceptions of the importance and frequency of technology-mediated language skills. Results show that student and instructor responses differ on certain technology-mediated skills, such as typing on a smartphone, underscoring the need to consider diverse perspectives in domain analysis research. Findings may inform how digital ELP admissions tests are developed, and how English for academic purposes curricula are designed, in order to better align test/classroom tasks with the academic language skills postsecondary students need.

A Rasch-based validation of the University of Tehran English Proficiency Test (UTEPT)

Recently, psychometricians and researchers have voiced their concern over the exploration of language test items in light of Messick’s validation framework. Validity has been central to test development and use; however, it has not received due attention in language tests having grave consequences for test takers. The present study sought to examine the validity of the University of Tehran English language proficiency test (UTEPT) using 6 aspects of Messick’s validation framework (i.e., content, structural, external, generalizability, substantive, and consequential). To examine each validity dimension, the Rasch model was considered. To this end, the data of 394 test takers who had taken the test in 2023 were cleaned and analyzed for the assumptions of the Rasch model and then for the 6 aforementioned validity aspects. The results indicated that the structural (i.e., unidimensionality), generalizability (i.e., measurement invariance), and substantive aspects held. However, the evidence for the content, external, and consequential aspects of validity was less definitive. The implications of the findings for the validity of the test, for the realm of language testing, and for item developers and item writers will be discussed.

The Academic Achievement of Undergraduate Students with Different English Language Proficiency Profiles

ABSTRACT Many English-medium universities employ a compensatory model to establish cutscores on English language proficiency tests for student admissions. In this model, students can have different scores on different sections of the test provided their overall score meets the admission cutscore. This practice raises questions regarding potential variation in academic achievement among students with diverse score profiles. To address these questions, this study compared the demographic characteristics and academic achievement of 3,694 undergraduate students who met the required cutscore on the IELTS-Academic for admission to a Canadian English-medium university but have different scores on different sections of the test. The findings indicated that, generally, students with medium reading scores combined with low or medium writing scores exhibited lower academic achievement compared to those with high scores on all the IELTS sections or high reading scores combined with medium or high writing scores. The profile groups demonstrated significant differences in certain demographic characteristics, potentially explaining why they have different proficiency levels in different language skills. The implications of the findings are discussed, including whether to maintain the compensatory model or switch to a mixed one, and the implications for providing English language support to students with diverse proficiency profiles.

Examining Performance on an Integrated Writing Task from a Canadian English Language Proficiency Test

Many English language proficiency (ELP) tests used for university admissions and placement now include integrated writing tasks that require examinees to use external sources when writing. Integrated writing tasks improve test authenticity and impact, but they raise several validity questions, such as what academic language skills they engage and whether performance on these tasks varies with examinee ELP level. This study addresses these questions with reference to an integrated writing task from the Canadian Academic English Language (CAEL) Test that involves reading, listening, and writing in academic contexts. Responses by 59 students to one of the CAEL integrated writing tasks are analyzed in terms of various grammatical, discourse, sociolinguistic, strategic, content, and source use aspects and compared across ELP levels (high and low) and score levels. The findings indicate that both ELP level and score level had significant effects on most writing features examined in the study, except for syntactic complexity. Additionally, except for syntactic complexity, all writing dimensions examined in the study were significantly associated with writing scores. The findings and their implications for the validity argument of source-based writing tasks are discussed.

Classification of multiple text reading test formats by analyzing English language proficiency tests

This study proposes a classification method for multiple text reading test formats in English language proficiency tests. A preliminary study involving 11 proficiency tests revealed two tests that fit the scope of the main study. Results show that multiple text reading test formats use complementary texts rather than conflicting texts. As for questions in a set of test forms in multiple text reading test formats, cognitive processing on integrating the contents of texts was different in proficiency tests. Moreover, the type of connection formation required by the questions differed among the proficiency tests. Implications for pedagogy are presented.

Investigating Passive Voice Problems in Kurdish Students’ Translation from and into English

Metaheuristic Translation is a means in which communication takes place between people who do not speak the same language and do not share the same culture. It is an undeniable fact that learners of language encounter problems in shifting a language into another using accurate aspects of the target one. This research looks at Badini-Kurdish students’ difficulties while translating passive voice structures from and into English. It also determines similarities and differences between English and Badini-Kurdish in terms of passive voice constructions. The participants of this study are 30 undergraduate students of the Department of Translation, College of Languages, University of Duhok. Two tests, i.e., an English language proficiency test and a translation test have been conducted for the purpose of data collection. Moreover, Larson’s (1984) model for translation assessment is used to assess the data obtained, combining statistical analysis with a qualitative evaluation. For data analysis, a descriptive method is used to explain the passive voice structure in both languages, a contrasted method is used to identify similarities and differences between the two languages, and a mixed method is used to accomplish the study’s goal. This research adds to the body of literature by illuminating the challenges and difficulties Badini-Kurdish students confront while translating passive voice constructs and offers insightful information for translators and language educators.

A Comparison and Evaluation of the TOEIC and DET Writing Test

This essay aims to compare and evaluate the writing test of two widely used English language proficiency tests: the Test of English for International Communication (TOEIC) and the Duolingo English Test (DET). Bachman and Palmer’s (2010) assessment use argument (AUA) is used as a framework in this essay to compare and evaluate these two tests from three aspects: test score and reliability, validity and score interpretations, and consequences. The results show that some features of the TOEIC and DET writing tests (e.g., the DET writing test’s well-designed algorithm and lack of discourse level texts, the TOEIC writing test’s all direct tasks and DIF analysis used) make the two tests have some strengths and weaknesses respectively in each aspect.

Mapping English language proficiency test scores of lecturers onto the common European framework level

This study was conducted to display the lecturer’s English language proficiency test scores at one of the state universities in Indonesia. The research applied descriptive design by quantitative approach. The sample is the lecturers of the English and non-English departments at one of the state universities in Indonesia. The instrument of the research was English language proficiency test of TOEFL ITP. Descriptive statistics was used as data analysis technique, mean and percentage. The result of this research disclosed that 86% of the lecturers are in the level of B1 of CEFR which achieved the targeted score of English competences certification file requirements of certified lecturer and 3.92% of the lecturers are in the A2 level of CEFR. The researchers discovered that the most difficult language skill of the test is in the structure and written expression section where 4 % of the lecturers are in the A2 CEFR level. Then it is also revealed that the lecturers got the lowest scaled score in the structure and written expression section, which was 37. From all parts of the test, none of the lecturers achieved C1 level in the structure and written expression. This is implied that the lecturers have problems in their writing skills and need to improve their grammatical skills to be able to write correct sentence structure in standard academic English.

Translanguaging pedagogies: Using an action research approach to support English as an Additional Language (EAL) students in a first-year undergraduate anatomy course.

Internationalisation of higher education has increased the number of EAL students studying in Anglophone countries, including Australia. Required English language proficiency tests (e.g., IELTS) do not reflect discipline-specific terminology nor prior learning necessary for successful study. Furthermore, anatomy study, the focus of this paper, requires an extensive terminology, predominantly derived from Latin or Greek, that creates significant learning challenge for all students, particularly EAL students.This pilot study was based in a large first-year anatomy course at an Australian university, with >30% EAL students. To address challenges for EAL students (as identified by teaching staff), translanguaging pedagogies were implemented through repeated cycles of action research. Translanguaging approaches, beyond simply translation of resources, employ EAL students’ full linguistic repertoire in the classroom to support acquisition of discipline content and the target language.Collaboration between the teaching team, a translanguaging expert and past EAL students identified specific translanguaging strategies for implementation. These strategies included a translanguaging workshop for teaching staff, encouragement of students’ use of home languages and targeted translated documents. A mixed methods approach was applied to evaluate the implementation. Initial findings show a positive response from both staff and students with additional reflections for future implementation and transferability to similar educational contexts.

The influence of English language proficiency test scores on the academic success of ESL undergraduate students

ABSTRACT This study considers language proficiency test scores achieved by ESL students who subsequently entered a New Zealand university at the undergraduate level. Scores from IELTS and the university’s in-house English Proficiency Test are analysed to determine the predictive ability of overall scores on these two tests and scores for each of the four language skills on academic success. The results indicated that language proficiency had a significant effect on undergraduate academic achievement. Furthermore, analyses of the predictive ability of scores for listening, speaking, and reading were remarkably similar between the tests, with non-significant results of similar magnitude for each skill. On the other hand, IELTS writing scores demonstrated a non-significant negative effect on academic achievement, while EPT writing scores were strongly significantly predictive of academic achievement. The results suggest that universities may benefit from measuring ESL students’ academic writing proficiency in more authentic university-level writing tasks.

Measuring Variability in Proctor Decision Making on High‐Stakes Assessments: Improving Test Security in the Digital Age

AbstractRemote proctoring, or monitoring test takers through internet‐based, video‐recording software, has become critical for maintaining test security on high‐stakes assessments. The main role of remote proctors is to make judgments about test takers' behaviors and decide whether these behaviors constitute rule violations. Variability in proctor decision making, or the degree to which humans/proctors make different decisions about the same test‐taking behaviors, can be problematic for both test takers and test users (e.g., universities). In this paper, we measure variability in proctor decision making over time on a high‐stakes English language proficiency test. Our results show that (1) proctors systematically differ in their decision making and (2) these differences are trait‐like (i.e., ranging from lenient to strict), but (3) systematic variability in decisions can be reduced. Based on these findings, we recommend that test security providers conduct regular measurements of proctors’ judgments and take actions to reduce variability in proctor decision making.

Examining the Impact of a Digital Reading Progress Tool on Saudi L2 Learners’ Reading Aloud Performance and Proficiency

: Few studies have considered the use of digital reading tools for improving the fluency and proficiency of Saudi learners of English as a second language (ESL). Hence, in this study, the researcher investigated the impact of Microsoft’s Digital Reading Progress tool on 30 Saudi ESL learners’ reading performance and proficiency. The participants were all at the B1 intermediate English language level in academic ESL, according to the Common European Framework of Reference for Languages (CEFR) English language proficiency test. The students were given one reading passage per week, and the researcher examined their performance in reading aloud over a period of four weeks using the Digital Reading Progress tool. The researcher employed a quantitative analysis, including post-assessments, to measure five aspects of participants’ reading performance: mispronunciation of words, self-corrections, omissions, repetitions, and correct words per minute. The findings revealed significant improvements in participants’ reading performance, fluency, and accuracy, as evidenced by increased accuracy after engaging with the Digital Reading Progress tool, increased scores across the five rated categories, and reduced pronunciation and word omission errors. This study contributes to the growing body of research on technology-enhanced language learning by demonstrating the positive impact of the Digital Reading Progress tool on L2 learners’ reading performance and proficiency. The findings have practical implications for ESL instructors in integrating digital tools as valuable pedagogical resources for enhancing L2 reading learning and performance.

The Use of TOEFL iBT® in Admissions Decisions: Stakeholder Perceptions of Policies and Practices

AbstractThis paper reports partial results from a larger study of how three different groups of stakeholders—university admissions officers, faculty in graduate programs involved in admissions decisions, and Intensive English Program (IEP) faculty—interpret and use TOEFL iBT® scores in making admissions decisions or preparing students to meet minimum test score requirements. Our overall goal was to gain a better understanding of the perceived role of English language proficiency in admissions decisions and the internal and external factors that inform decisions about acceptable ways to demonstrate proficiency and minimal standards. To that end, we designed surveys for each stakeholder group that contained questions for all groups and questions specific to each group. This report focuses on the questions that were common to all three groups across two areas: (1) understandings of and participation in institutional policy making around English language proficiency tests and (2) knowledge of and attitudes toward the TOEFL iBT test itself. Our results suggested that, as predicted, university admissions staff were the most aware of and involved in policy making but frequently consulted with ESL experts such as IEP faculty when setting policies. This stakeholder group was also the most knowledgeable about the TOEFL iBT test. Faculty in graduate programs varied in their understanding of and involvement in policy making and reported the least familiarity with the test. However, they reported that more information about many aspects of the test would help them make better admissions decisions. The results of the study add to the growing literature on language assessment literacy among various stakeholder groups, especially in terms of identifying aspects of assessment literacy that are important to different groups of stakeholders.

Differential emotional expression in autobiographical narratives: The case of Arabic–English bilinguals

Aims and Objectives: This study aimed to examine how bilinguals express their emotions in their first language (L1) versus their second language (L2) in autobiographical narratives. Methodology: A total of 64 Egyptian learners of English as a foreign language (EFL) took an English language proficiency test, and wrote about the happiest or saddest experiences they recalled in Arabic and in English. Data and Analysis: The narratives were analysed for the frequency and the valence of emotion words. Findings/Conclusions: The results showed no difference between the narratives of the two languages in terms of the frequency of words. However, the L2 narratives were generally more negatively valenced than the L1 narratives, and more emotional polarity was noted in the L2 narratives, particularly in sad situations. The results failed to show any effect for the participants’ age, gender or L2 proficiency on the frequency or valence of the emotion words. Originality: This study compared two languages that are distant at several levels, including typography and culture. In addition, the study examined some variables that were not investigated in earlier emotion studies of bilingualism. Significance/Implications: The study presents several implications in relation to the emotional contexts of learning theory and the pedagogy of emotion words.

Predictors of English Medium Instruction academic success in Vietnamese Higher Education

Abstract Recent research has investigated predictors of English Medium Instruction (EMI) academic success in Higher Education in various contexts; however, mixed findings have been reported, and empirical research in emerging contexts like Vietnam is relatively scarce. In addition, little is known about the associations between tertiary EMI outcome, gender, and high school performance. To fill these research gaps, the present study was conducted to explore the roles of gender, high school grades, English language proficiency and vocabulary knowledge (both receptive and productive knowledge) in Vietnamese tertiary EMI academic achievement. To this end, 78 Vietnamese undergraduates majoring in business studies participated in the study. The participants took the Vocabulary Levels Test (Schmitt et al. 2001) and Productive Vocabulary Levels Test (Laufer and Nation 1999) to measure their receptive and productive vocabulary knowledge. Their English language proficiency test scores, EMI course grades, and high school grade point average (GPA) were also collected for data analysis. Simple linear regression analyses revealed that high school GPA, English language proficiency together with both receptive and productive vocabulary knowledge significantly predicted EMI academic outcome while gender did not. The results are discussed with implications for EMI in Higher Education.

Practical considerations when building concordances between English tests

Applicants must often demonstrate adequate English proficiency when applying to postsecondary institutions by taking an English language proficiency test, such as the TOEFL iBT, IELTS Academic, or Duolingo English Test (DET). Concordance tables aim to provide equivalent scores across multiple assessments, helping admissions officers to make fair decisions regardless of the test that an applicant took. We present our approaches to addressing practical (i.e., data collection and analysis) challenges in the context of building concordance tables between overall scores from the DET and those from the TOEFL iBT and IELTS Academic tests. We summarize a novel method for combining self-reported and official scores to meet recommended minimum sample sizes for concordance studies. We also evaluate sensitivity of estimated concordances to choices about how to (a) weight the observed data to the target population; (b) define outliers; (c) select appropriate pairs of test scores for repeat test takers; and (d) compute equating functions between pairs of scores. We find that estimated concordance functions are largely robust to different combinations of these choices in the regions of the proficiency distribution most relevant to admissions decisions. We discuss implications of our results for both test users and language testers.

Interpretation and Use of a Workplace English Language Proficiency Test Score Report: Perspectives of TOEIC® Test Takers and Score Users in Taiwan

AbstractResearch in validity suggests that stakeholders' interpretation and use of test results should be an aspect of validity. Claims about the meaningfulness of test score interpretations and consequences of test use should be backed by evidence that stakeholders understand the definition of the construct assessed and the score report information. The current study explored stakeholders' uses and interpretations of the score report of a workplace English language proficiency test, the TOEIC® Listening and Reading (TOEIC L&R) test. Online surveys were administered to TOEIC L&R test takers and institutional and corporate score users in Taiwan to collect data about their uses and interpretations of the test score report. Eleven survey respondents participated in follow‐up interviews to further elaborate on their uses of the different score reporting information within the stakeholders' respective contexts. Results indicated that the participants used the TOEIC L&R test scores largely as intended by the test developer although some elements of the score report appeared to be less useful and could be confusing for stakeholders. Findings from this study highlight the importance of providing score reporting information with clarity and ease to enhance appropriate use and interpretation.

Test review: High-stakes English language proficiency tests—Enquiry, resit, and retake policies

Many candidates undertaking high-stakes English language proficiency tests for academic enrolment do not achieve the results they need for reasons including linguistic unreadiness, test unpreparedness, illness, an unfavourable configuration of tasks, or administrative and marking errors. Owing to the importance of meeting goals or out of a belief that original test performance was satisfactory, some individuals query their results, while others go on to retake the test, perhaps on multiple occasions. This article critically reviews the policies of eight well-known, on-demand gatekeeping English language tests, describing the systems adopted by language assessment organisations to regulate results enquiries, candidates resitting (components of) a test where performance fell short of requirements, and repeat test-taking. It was found that all providers institute clear mechanisms through which candidates can query their results, with notable variations exhibited in procedures, costs, restrictions, outcomes, and how policies are communicated to test-takers. Test resit options are scarce, while organisations enact few restrictions on test retakes in the form of mandatory waiting times and cautionary advice. The implications for language assessment organisations are discussed.

Item analysis of reading comprehension questions for English proficiency test using Rasch model

The need to take English as a foreign language proficiency test (known as TOEFL [Test of English Language Proficiency]) has been gaining popularity in Indonesia. The increasing demands for such a test and its expensive cost have reinforced many institutions to develop TOEFL instruments and administer the test internally. However, constructing a test instrument is a complex process that makes conducting item analysis become more challenging. Meanwhile, item analysis is crucial to assess the items’ quality. Therefore, this study reported the results of statistically analyzing 20 questions of TOEFL reading comprehension that were analyzed in terms of the test reliability, the item and person fit, and the items’ difficulty level. Thirty-eight members of the English Department Students’ Association of a state university in West Java participated in this study by taking the reading test. The data were analyzed using the Rasch model by utilizing the Quest program. The results showed that four items (36.8%) did not fulfill the ideal criteria of a valid test because they were too easy and too difficult to be given to the target test takers; thus, they needed to be discarded. Meanwhile, 16 items (63.2%) are of good quality and can be used immediately in the proficiency test, especially to measure reading comprehension skills, because they have fulfilled the standard requirements for a valid test. The findings have provided insight into the importance of item analysis in validating test instruments to improve the test quality for future administrations.

Toward a systematic accessibility review process for English language proficiency tests for young learners

In order to make assessments as widely accessible as possible, including to young learners from diverse backgrounds with a wide range of individual needs and characteristics, some developers of standardized tests have resorted to offering accessibility tools (e.g., magnifying/zoom) and accommodations (e.g., extended response time) to test takers. These solutions have some limitations stemming from the fact that they are retroactively applied to tests. One possible avenue to meet the accessibility needs of diverse students is to proactively address accessibility in the development of test content for all learners. In this article, we report on the development and piloting of a systematic accessibility review process intended for all test content at an organization producing large-scale standardized English proficiency tests for elementary and secondary education students in the United States. We describe the theoretical and research foundations related to fairness, accessibility, and universal design that guided our development of the accessibility review process followed by the process used to develop the tool. The application of the accessibility checklist to a kindergarten English language proficiency assessment is described in detail. Finally, we share some considerations for how the accessibility checklist could be used in other assessment development contexts.