TOEFL iBT Research Articles

Psychometric Characteristics of a Rating Scale for Assessing Interactional Competence in Paired-Speaking Tasks at Micro-level

Developing rating scales for assessing interactional performance is demanding since it is a relatively complicated procedure. The present study investigated the psychometric characteristics of the CAP rating scale (Wang, 2015) for assessing interactional competence at micro-level. To this end, 160 Iranian intermediate EFL learners were selected based on their performance on TOEFL iBT test from a language institute in Tabriz. Four interaction tasks were used to elicit students’ performance on interactional competence using the CAP rating scale. Five raters were recruited in the study to assign score to each individual’s performance. The participants were pretested and post-tested at the beginning and the end of the term through the same scale. The Pearson correlations were computed in order to estimate the test-retest reliability indices of the scale. In addition, five separate exploratory factor analysis (EFA) through the varimax rotation method were conducted in order to investigate the underlying constructs of the communication functions individually and as a total. The results revealed that the CAP rating scale enjoys a reasonable reliability indices and the four functions i.e. building argument, developing discussion, offering support, and shaping connection can be appropriate predictors of interactional competence. Some pedagogical and assessment implications are presented as well.

Holistic and Analytic Assessments of the TOEFL iBT® Integrated Writing Task

Holistic and Analytic Assessments of the TOEFL iBT® Integrated Writing Task

Operationalization of the Speaking Construct in the Internet-Based Test of English as a Foreign Language

In recent years, tests of speaking are integral parts to standardized language assessments because test users expect candidates to be able to communicate effectively in a foreign language, for a variety of different purposes and in variety of different modes (Powers 2010:1). The Internet-based Test of English as a Foreign Language is designed so as to address a number of language purposes using all four language skills, independently or in integration. In this paper we use a modified version of Bachman and Palmer’s test task characteristic framework (1996) to analyze speaking tasks in the Speaking Section of TOEFL iBT, identifying speaking purposes the test addresses through the process of construct operationalization. The findings help us reflect on the authenticity of test tasks, as well as on the correspondence between test tasks and speaking tasks in target language use domains. In addition, task analysis reveals the test’s limitations concerning situational/interactional authenticity due to non-live mode of test delivery.

INVESTIGATING THE USE OF WEB 2.0 TOOLS FOR TOEFL IBT SPEAKING PRACTICE TEST

The use of independent and integrated speaking tasks represents a distinctive element of the TOEFL iBT speaking exam. Integrated tasks that involve synthesizing and summarizing information presented in reading and listening materials have the potential to generate new test preparation strategies. Language teachers, whether in schools or colleges, have started using Web 2.0. tools in order to prepare students for language exams. It is asserted that Web 2.0. tools support active and meaningful learning and help students to express themselves on a particular subject. This paper describes the use of Web 2.0. tools to simulate TOEFL iBT-style speaking exercises and improve the students’ test taking ability in speaking during a course for kindergarten teachers in Andychów. Web-based activities were also implemented to help students overcome their fear of speaking. Also discussed are the main problems encountered, both pedagogical and technical, and what was done to solve them. Student feedback from an end-of-the-semester survey and from qualitative interviews is shared. The survey study shows that the use of Web 2.0. tools was a key feature of exam preparation on this intensive course. It is agreed that speaking skills of students can be improved through careful implementation of Web 2.0. tools.

An Investigation of the Predictive Validity of the TOEFL iBT® Test at an English‐Medium University in Turkey

This study reports on an investigation of the predictive validity of the TOEFL iBT® test in an English‐medium institution (EMI) in a non‐target‐language context, namely, Turkey. The relationship between TOEFL iBT scores and academic performance was explored in a cohort of 286 undergraduate students, as was the TOEFL iBT's relationship with an institutional English proficiency exam (Certificate of Proficiency in English [COPE]) used as a benchmark for faculty entry. Performance measures included scores on TOEFL iBT and COPE, grade point averages (GPAs) for content and English for academic purposes (EAP) courses over 2 freshman semesters, and freshman language instructor evaluations of students' freshman EAP performance. Correlations between test scores confirmed a moderate to moderately high predictive validity for content course GPAs and English‐language course GPAs, respectively, and for the TOEFL iBT, particularly in technical fields. Instructor evaluations of student performance supported the findings, with fewer deficiencies in academic English skills for students with higher scores on TOEFL iBT. The study concludes that the TOEFL iBT's predictive validity is on par with the institution's own proficiency test and represents a solid performance measure for use in EMIs in a non‐target‐language context.

Domain Description: Validating the Interpretation of the TOEFL iBT® Speaking Scores for International Teaching Assistant Screening and Certification Purposes

In the past 2 decades, there has been an increasing tendency to use scores from the TOEFL iBT® Speaking test for decisions regarding the certification of international graduate students as teaching assistants at North American universities. To obtain validity evidence in support of the usefulness of the speaking scores for this secondary use of the test, this study adopted the argument‐based approach to validation. Focusing on the domain description inference in the TOEFL interpretive argument, the study investigated whether the language functions elicited by TOEFL iBT Speaking tasks can be identified in authentic discourse produced by international teaching assistants (ITAs) with different instructional roles. We compiled and analyzed 2 corpora—a TOEFL iBT speech corpus and an ITA speech corpus. The TOEFL corpus contained 2,738 responses to integrated and independent tasks. The ITA corpus of 119 spoken texts included multiple disciplines and 3 instructional genres: lab, recitation, and lecture. The 2 corpora were manually annotated using the knowledge framework (Mohan, 1986), which is a heuristic in systemic functional linguistics (SFL) used to identify knowledge structures (KSs) and language functions based on how linguistic choices function in the discourse. Then, the following types of data were quantitatively analyzed: discourse units annotated per KS category, discourse units annotated per language function, KS categories occurring in each text, language functions occurring in each text, and KSs and functions from each component of the spoken corpora. The corpus data revealed how the language functions were realized and how they varied. Overall, the results indicated that TOEFL iBT Speaking tasks elicit most of the language functions identified in ITA discourse, suggesting that this test accounts for the functional language ability necessary for effective instructional performance as a teaching assistant in the target domain of language use. The discrepancy detected in the use of some functions pertaining to 2 KSs warrants further examination of the extent to which it may impact secondary test use and score interpretation.

TEST TAKERS’ ATTITUDES TO THE TEST CONTENT OF THE TWO LISTENING TESTS: IELTS AND TOEFL iBT

Ten aspects of test content in the two listening tests: IELTS and TOEFL iBT are investigated from the perspective of test-takers’ judgment. Main findings reveal that there are both similarities and differences in test takers’ attitudes to the two tests although the similarities outweigh the differences. The most obvious difference is that test takers have a more positive attitude to the IELTS listening test than to the TOEFL iBT listening test and test preparation has a strong effect to test takers’ attitude to the test. In addition, test takers’ positive attitudes to the test are strongly associated with better test performance. Substantial differences of test takers’ attitude to the two listening tests can be seen in their judgment of difficulty level, new words/technical terms and familiarity of topics. Test takers found the IELTS listening test less difficult, having fewer new words and technical terms, and containing more familiar topics than the TOEFL iBT listening test. They also find the test method of the IELTS is less challenging than that of the TOEFL iBT listening test although their choice of the test to take heavily depends on which test they are being prepared for.

The Effects of International Accents and Shared First Language on Listening Comprehension Tests

This study examines the effect of incorporating a variety of international English accents into a simulated TOEFL listening comprehension test in growing recognition of internationalization of language teaching and learning in the field of TESOL. Although some high‐stakes English proficiency exams have begun incorporating speech samples produced by speakers from a range of inner circle English‐speaking backgrounds (e.g., Britain, the United States, Australia), the inclusion of samples produced by speakers of outer and expanding circle English varieties (e.g., India, Nigeria, Mexico, South Korea) has been largely avoided. For this study the researchers recruited speakers from six distinct English varieties to produce speech samples for a mock TOEFL iBT listening exam. Listeners who spoke with the same six international English accents were then recruited to take the resulting tests. Results indicate that when accented English is highly comprehensible, listening test scores for stimuli based on high‐proficiency speakers of outer and expanding circle varieties of English are not significantly lower than they are in response to stimuli based on inner circle varieties of English. With respect to a shared first language effect on test scores when test materials are spoken in the test taker's own accent, results are complex but inconclusive.

A quantitative analysis of TOEFL iBT using an interpretive model of test validity

BackgroundValidity is a notable concept in language testing which has concerned many researchers and scholars in the field of language testing due to its importance in decision making process. Tests’ results always introduce consequences to test takers’ lives which emphasizes the need to ensure their validity. Detecting and delineating the potential sources that may threaten the validity of standardized tests of English proficiency is therefore of great importance.MethodsThis study was a quantitative investigation of test validity using an interpretive model. The main purpose of this study was to quantitatively assess and determine the relationship between a series of inferences in an interpretive argument-based framework that can potentially threaten the validity of the test.ResultsTo this end samples of TOEFL iBT were analyzed and the obtained results determined its validity expressed in terms of the chain of inferences in the interpretive model.ConclusionThe findings indicated that there is a direct relationship between the performance on the TOEFL iBT and the target domain of English language use, observed test scores are reflective of intended academic language abilities, observed test scores can be generalized to similar language tasks in different occasions and test forms, and expected test scores can be accounted for by underlying language abilities in an academic environment.

The Effects of 'Face' on Listening Comprehension: Evidence from Advanced Jordanian Speakers of English.

This empirical study examines the extent to which 'face', i.e. (audio visual dialogues), affects the listening comprehension of advanced Jordanian EFL learners in a TOFEL-like test, as opposed to its absence (i.e. a purely audio test) which is the current norm in many English language proficiency tests, including but not limited to TOFEL iBT, TOEIC and academic IELTS. Through an online experiment, 60 Jordanian postgraduate linguistics and English literature students (advanced EFL learners) at the University of Jordan sit for two listening tests (simulating English proficiency tests); namely, one which is purely audio [i.e. without any face (including any visuals such as motion, as well as still pictures)], and one which is audiovisual/video. The results clearly show that the inclusion of visuals enhances subjects' performance in listening tests. It is concluded that since the aim of English proficiency tests such as TOEFL iBT is to qualify or disqualify subjects to work and study in western English-speaking countries, the exclusion of visuals is unfounded. In actuality, most natural interaction includes visibility of the interlocutors involved, and hence test takers who sit purely audio proficiency tests in English or any other language are placed at a disadvantage.

Comparability of students’ writing performance on TOEFL iBT and in required university writing courses

Investigating the comparability of students’ performance on TOEFL writing tasks and actual academic writing tasks is essential to provide backing for the extrapolation inference in the TOEFL validity argument (Chapelle, Enright, & Jamieson, 2008). This study compared 103 international non-native-English-speaking undergraduate students’ performance on two TOEFL iBT® writing tasks with their performance in required writing courses in US universities as measured by instructors’ ratings of student proficiency, instructor-assigned grades on two course assignments, and five dimensions of writing quality of the first and final drafts of those course assignments: grammatical, cohesive, rhetorical, sociopragmatic, and content control. Also, the quality of the writing on the TOEFL writing tasks was compared with the first and final drafts of responses to written course assignments using a common analytic rubric along the five dimensions. Correlations of scores from TOEFL tasks (Independent, Integrated, and the total Writing section) with instructor ratings of students’ overall English proficiency and writing proficiency were moderate and significant. However, only scores on the Integrated task and the Writing section were correlated with instructor-assigned grades on course assignments. Correlations between scores on TOEFL tasks and all dimensions of writing quality were positive and significant, though of lower magnitude for final drafts than for first drafts. The TOEFL scores were most highly correlated with cohesive and grammatical control and had the lowest correlations with rhetorical organization. The quality of the writing on the TOEFL tasks was comparable to that of the first drafts of course assignment but not the final drafts. These findings provide backing for the extrapolation inference, suggesting that the construct of academic writing proficiency as assessed by TOEFL “accounts for the quality of linguistic performance in English-medium institutions of higher education” (Chapelle, Enright, & Jamieson, 2008, p. 21).

Strategy Use in the TOEFL iBT Speaking Test and Academic Classroom

Strategy Use in the TOEFL iBT Speaking Test and Academic Classroom

Strategy Use in the TOEFL iBT Speaking Test and Academic Classroom

Strategy Use in the TOEFL iBT Speaking Test and Academic Classroom

Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing

Abstract This study develops a model of second language (L2) writing quality in the context of a standardized writing test (TOEFL iBT) using a structural equation modeling (SEM) approach. A corpus of 480 test-takers’ responses to source-based and independent writing tasks was the basis for the model. Four latent variables were constructed: an L2 writing quality variable informed by scores of source-based and independent writing tasks, and lexical sophistication, syntactic complexity, and cohesion variables informed by lexical, syntactic, and cohesive features within the essays. The SEM analysis showed that an L2 writing quality model had a good fit, and was generalizable across writing prompts (with the exception of lexical features), gender, and learning contexts. The structural regression analysis indicated that 81.7% of the variance in L2 writing quality was explained by lexical decision reaction time scores (β = 0.932), lexical overlap between paragraphs (β = 0.434), and mean length of clauses via lexical decision reaction time scores (β = 0.607). These findings indicate that higher-rated essays tend to contain more sophisticated words that elicited longer response times in lexical decision tasks, greater lexical overlap between paragraphs, and longer clauses accompanying more sophisticated words. Implications for evaluating lexical, syntactic, and cohesive features in L2 writing are discussed.

Using Corpus‐Based Register Analysis to Explore the Authenticity of High‐Stakes Language Exams: A Register Comparison of TOEFL iBT and Disciplinary Writing Tasks

AbstractOne of the central considerations in the validity argument for the TOEFL iBT is the relationship between the language on the exam and the language required for university courses. Corpus linguistics has recently been shown to be an effective way to explore this relationship, which can also be considered as an aspect of authenticity. Applying Multi‐Dimensional Analysis, our study is the first to directly compare the lexico‐grammatical characteristics of texts produced by second language writers on a high‐stakes standardized exam (the TOEFL iBT) to the characteristics of texts produced by the same writers for disciplinary writing tasks required as part of their academic career (e.g., lab reports, argumentative essays). Our findings show that the language of TOEFL iBT tasks have patterns of both similarity and difference from disciplinary tasks with respect to four linguistic dimensions of language use: (a) Compressed Procedural Information versus Stance toward the Work of Others, (b) Personal Stance, (c) Possible versus Completed Events, and (d) Information Density. Integrated iBT tasks show similarities to certain kinds of disciplinary tasks, while independent iBT tasks differ linguistically from almost all disciplinary writing tasks. These general patterns hold regardless of the score levels of test takers. We discuss the implications of these findings for language teaching and assessment.

The Effect of CALL-based Instruction on Students' Score Attainment on the TOEFL iBT in a Yemeni Context

This article is an action research of quasi-experimental nature. It aims at investigating the effect of CALL-based instruction on Yemeni students' score attainment on the TOEFL iBT. The sample of study consisted of 30 adult students randomly assigned to experimental and control groups. Data was collected within an 8-week period via a pretest-posttest design for equivalent groups. The experimental group participants were also interviewed to elicit data about their attitudes towards CALL. The findings of the study indicated that there was a statistically significant difference between the experimental group (trained with CALL) and the control group (trained conventionally) in the total gain scores, as well as in the section gain scores of reading, listening, speaking and writing. The findings also showed that the participants in the experimental group held positive attitudes towards CALL. Therefore, it is highly recommended that CALL should be fully integrated into TOEFL iBT preparation programmes.

A Comparison of TOEFL iBT and IELTS Reading Tests

With the increasing popularity among test takers and mounting acceptance from academic institutions, TOEFL iBT and IELTS are regarded and treated as equals due to similar purposes and goals. In depth, they distinguish each other in various aspects. By integrating Bachman’s framework of test method facet and Alderson’s variables of reading nature, this article formulates a comparison framework in an attempt to compare the reading parts of these two tests. The results show they share more similarities in test rubrics and input format and slightly differ from each other in score method and specification of procedures and tasks. However, substantial differences are identified in such areas as the nature of language and characteristics of tasks, which has a significant impact on test takers’ performance because TOEFL iBT reading test is more difficult in terms of readability, text types, topical features and question types, while IELTS more complex in text length and grammatical intricacy.

The Effect of CALL-Based Instruction on Students' Score Attainment on the TOEFL iBT in a Yemeni Context

This article is an action research of quasi-experimental nature. It aims at investigating the effect of CALL-based instruction on Yemeni students' score attainment on the TOEFL iBT. The sample of study consisted of 30 adult students randomly assigned to experimental and control groups. Data was collected within an 8-week period via a pretest-posttest design for equivalent groups. The experimental group participants were also interviewed to elicit data about their attitudes towards CALL. The findings of the study indicated that there was a statistically significant difference between the experimental group (trained with CALL) and the control group (trained conventionally) in the total gain scores, as well as in the section gain scores of reading, listening, speaking and writing. The findings also showed that the participants in the experimental group held positive attitudes towards CALL. Therefore, it is highly recommended that CALL should be fully integrated into TOEFL iBT preparation programmes.

Which Should Be Dominant in Japan, the TOEFL iBT or the IELTS? (Course name: What Does a Test Measure?)

Which Should Be Dominant in Japan, the TOEFL iBT or the IELTS? (Course name: What Does a Test Measure?)

The Contribution of Lexical Diversity to College‐Level Writing

This article reports on a study that investigated the extent to which lexical frequency and lexical diversity contribute to writing proficiency scores on monolingual English‐speaking writers’ and advanced multilingual writers’ academic compositions. The data consist of essays composed by 104 multilingual English learners enrolled in advanced second language writing courses at various intensive English programs and 68 monolingual English‐speaking university students in a first‐year composition course. Three independent raters evaluated the essays according to the TOEFL iBT independent writing rubric. Results from a binary logistic regression reveal that lexical diversity has a significantly greater impact on writing score than lexical frequency. Post hoc analyses show that lexical frequency has only a moderate relationship to lexical diversity, suggesting that diversity of mid‐frequency vocabulary may be more important to writing proficiency than the use of more sophisticated terms that occur less frequently in natural language. Implications for practice suggest that it is not enough to simply teach vocabulary words in the second language composition classroom, but also to guide learners in how to employ these words in a varied manner in their writing.