Discrete-point Tests Research Articles

ISTORIJA JEZIČKOG TESTIRANjA

The paper presents the history of the development of language test- ing in the light of various, once prevailing theories about the nature, acquisition and use of language. The initial chapter is dedicated to the first, traditional or pre-scientific phase in the development of language testing, which is related to the end of the 19th century and the then dominant grammar-translation method. What follows is a presentation of the period of discrete point testing, resulting from a structuralist interpretation of language, as well as the attempt to overcome the shortcomings of this type of testing. The final chapters deal with the theory of communicative competence, which has led to an essential and final turning point in relation to traditional methods. We also paid attention to the new challenges that the language testing process faced in the conditions of a pandemic and a complete transition to online teaching, and the answers to the questions that were recognized as problematic could be given by some future research.

Tahlīl Muhtawā wa Mustawā As'ilat Ikhtibārāt Al-Lughah Al ‘Arabiyah Li Imtihān Al Madrasah 'alā al Ma'āyīr Al Wathaniyyah 'alā Ithār Al Manhaj 2013

Abstract
 The 2013 curriculum demands a paradigm shift in learning, process, and evaluation, including in HOTS, MOTS, LOTS-based assessments. Using an evaluative research design, this study aims to analyze the Arabic language test items in the UAMBN Madrasah Aliyah under LP. Ma'arif Kabupaten Jepara for the Academic Year 2018/2019. The analysis includes the design and approach used in preparing the test, the content (materials) of the test, and the categorization of the quality level of the test items in the 2013 Curriculum, i.e. HOTS, MOTS, or LOTS. The results showed that: first, the approach used in preparing the test was an integrative approach for multiple-choice test questions and a discrete-point test approach for essay test questions. Second, the content of the test included Arab and Indonesian culture and civilization, Muslim figures, language level, language rules, and sentence structure. Third, Arabic language test questions in UAMBN covered HOTS, MOTS, and LOTS questions. In the multiple-choice questions, 11 questions (27.5%) were included in the LOTS category, 10 questions (25%) in the MOTS category, and 19 questions (47.5%) in the HOTS category. Meanwhile, the essay questions, which presented 4 questions (80%) for the HOTS category and 1 question (20%) for the MOTS category, were considered less proportional. Overall, the format for the Arabic test in the UAMBN Madrasah Aliyah has been proportional because it has accommodated questions with the HOTS, MOTS, and LOTS categories.

Linking Learners’ Perspectives on Language Assessment Practices to Teachers’ Assessment Literacy Enhancement (TALE): Insights from Four European Countries

ABSTRACT This article presents results from a needs analysis survey conducted in the first year of a European-funded project entitled ‘Teachers’ Assessment Literacy Enhancement (TALE)’. The survey questionnaire used asked 1788 learners of English in Cyprus, Germany, Greece and Hungary about their experiences of assessment; which of these they considered conducive to learning and the role feedback played as an instrument of formative assessment. Further questionnaire data from their 658 teachers were included in the data analysis. The results showed that practices differed across contexts. Overall, both learners and teachers reported a wide range of skills and areas to be assessed in the EFL classroom with writing, followed by speaking, being assessed the most. Based on the perceptions reported by the learners, the assessment types used revealed rather traditional approaches with frequent use of e.g. discrete-point tests with closed answers, extended writing and translation. The learners appeared to regard these types of assessment to be supportive of their learning. Feedback given was mostly restricted to marks and brief comments. The perceptions on feedback practices varied among teachers and their learners. Results of the needs analysis were taken as the basis of the online course design for enhancing teachers’ language assessment literacy.

The effect of task sequencing on second language written lexical complexity, accuracy, and fluency

Planning and sequencing lessons in task-based language teaching has been considered a challenge because there is no agreed-upon theoretical framework for sequencing tasks. The SSARC model (stabilize, simplify, automatize, reconstruct, and complexify) of task sequencing allows for predictions about L2 learners’ interlanguage development. However, previous task sequencing studies have reported mixed findings. Therefore, the current study further investigated the effectiveness of sequencing tasks using the SSARC model for promoting L2 written lexical complexity (e.g., lexical diversity), grammatical accuracy, and fluency. Novice French L2 writers (N = 42) at an elementary school in Lebanon carried out three experimental tasks in two different orders: simple-to-complex, and complex-to-simple. Pre-test and post-test measures of their written production and discrete point tests of target lexical and grammatical forms were administered. Results showed that the writers in both groups improved in terms of their lexical diversity, grammatical accuracy of relative clauses, and fluency, but the simple-to-complex group maintained the gains over time. Pedagogical implications for task sequencing in L2 contexts are discussed.

The language assessment literacy needs of Iranian EFL teachers with a focus on reformed assessment policies

Teachers’ assessment literacy has recently captured the attention of scholars across various educational contexts. The literature has it that there is a gap between teachers’ assessment practices and national assessment policies. The present study investigated the assessment needs of Iranian EFL teachers in the wake of the new assessment reform, which aims at replacing traditional discrete point testing policies with performance testing. In-depth interviews were conducted with 15 EFL head teachers. In addition, documents related to the curriculum reform were also closely examined. Inductive coding of the data showed that to meet the demands of the noted reform, teachers’ current perceptions of language assessment need to change. Furthermore, teachers need training in both knowledge and skills of language assessment. More specifically, teachers need training in developing rubrics for use in assessing the productive skills of speaking and writing. They also need to develop literacy in devising higher-order thinking skills in assessing reading and listening comprehension. Finally, as non-native speakers of English, Iranian English teachers need better English aural/oral skills.

Tapping the Potential of Skill Integration as a Conduit for Communicative Language Teaching

The purpose of this classroom-based study was to discover the kinds of skill integration tasks that were employed by English teachers in Kuwait and to measure their attitudes toward implementing the skill integration technique in their classrooms. Data collection involved recording 25 hours of classroom-based observations, conducting interviews with the same group of teachers, and distributing a survey to further explore the teachers’ attitudes toward the skill integration technique. Data analysis involved categorizing skill integration tasks, analyzing the interview data, and counting the means and standard deviations of the survey data. Findings indicated that the participating teachers performed a wide range of transactional and interactional tasks that involved the simultaneous integration of speaking, listening, reading, and writing in their classrooms. The findings also revealed that even though the skill integration technique was adopted by most of the English teachers, they were ambivalent toward its implementation in their classrooms. This was partly due to the negative washback effect of traditional English tests that measure students’ accurate application of grammar rules but not their fluency and ability to use the L2 as a tool for communication. Implications for L2 pedagogy were drawn regarding the need for teachers to expose students of all proficiency levels to both transactional and interactional tasks in the classroom. To counter the negative washback effect of conventional discrete-point tests, English teachers were encouraged to develop communicative tests that involve skill integration and emphasize the development of the four language skills in their daily classroom activities.

Assessing Yemeni EFL learners’ Oral skills via the Conceptualization of Target Language Use Domain: A Testing Framework

There is an evident lack of a comprehensive evaluation basis for Yemeni learners’ speaking skills in the English department, Hodeidah University. The present paper presents a detailed framework of oral assessment criteria that involves a description of target language use domains and then shows how such domains can be systematically related to test design. The framework takes as its main goal the development and description of a criterion referenced rating scale representing real-world criterion elements. The aim of the testing framework, therefore, is to ensure maximum appropriateness of score test interpretations and maximize the validity and fairness of local speaking tests. A five-point likert scale is carried out to elicit 10 trained raters’ perceptions of using the pilot scale. The research findings support the use and appropriateness of the scale as it aids raters identify underlying aspects of their learners’ oral discourse that cannot be observed in traditional discrete point tests.

Context-Based Online Diagnostic Test of Spanish

Among the areas of emphasis in the language-teaching profession today we find considerable emphasis on (a) evaluation, (b) contextualization, and (c) technology. Although the greater emphasis in language assessment has been on global proficiency, there is also a need to examine specific points of students' language competence in order to help them improve the correctness of their communication. Considering these areas of emphasis, we undertook an effort to create a contextualized, online, diagnostic test of Spanish. This article explains the rationale behind the project and the procedures for accomplishing the task. We also describe a study conducted to determine the validity and practicality of the contextualized test. Results of the study indicate there appears to be little, if any, measurement advantage in using a contextualized test over a more traditional discrete-point test.

On Validation of Concept Map as an Assessment Tool of L2 Reading Comprehension: A Triangulation Approach

This study aims at addressing the issues related to reliability and validity of adoption of concept maps to assess L2 reading comprehension. To this end, this study was designed in two major phases. During the first phase, a test of reading in the form of select and fill-in concept map was designed, and its reliability (KR-21), content, concurrent (IELTS), and construct validity (using protocol analysis) were substantiated. During the second phase 300 participants were asked to take the newly-made test along with two other test formats (multiple choice and selective gapped summary) to determine which one is more correlated with L2 reading proficiency. The results demonstrated that concept map can be a reliable and valid tool of L2 reading assessment, having the potential to assess discourse comprehension and connected understanding and making reconciliation between objective discrete-point tests and integrative ones.

A Close Look at the Relationship between Multiple Choice Vocabulary Test and Integrative Cloze Test of Lexical Words in Iranian Context

In spite of various definitions provided for it, language proficiency has been always a difficult concept to define and realize. However the commonality of all the definitions for this illusive concept is that language tests should seek to test the learners’ ability to use real-life language. The best type of test to show such ability is considered to be the integrative test and cloze test is in turn regarded to constitute nearly all the factors needed for language use ability. However, the greatest obstacle of cloze tests or in general pragmatic tests is their administrative and scoring constraints for a large number of testees. Discrete-point tests, as the easiest and most common type of tests used for valid national and international proficiency tests, have always been doubtfully questioned as to whether they indicate the learners’ ability to use language in real-life situation, but due to their tangible shortcomings, no absolute answer can be provided. The study aims at shedding light at the idea of the extent to which the discrete items of vocabulary proficiency show the learners’ vocabulary proficiency in the real world of language use. Hence, the study seeks to calculate the correlation between discrete-point and integrative language proficiency tests of vocabulary administered to 21 Iranian freshmen studying English as a Foreign Language.

The Predominance Of Integrative Tests Over Discrete Point Tests In Evaluating The Medical Students' General English Knowledge

Background and purpose: Multiple choice tests are the most common type of tests used in evaluating the general English knowledge of the students in most medical universities, however the efficacy of these tests are not examined precisely. We compare and examine the integrative tests and discrete point tests as measures of the English language knowledge of medical students. Methods: Three tests were given to 60 undergraduate physiotherapy and Audiology students in their second year of study (after passing their general English course). They were divided into 2 groups. The first test for both groups was an integrative test, writing. The second test was a multiple – choice test of prepositions for group one and a multiple – choice test of tenses for group two. The same items which were most frequently used wrongly in the first test were used in the items of the second test. A third test, a TOEFL, was given to the subjects in order to estimate the correlation between this test and tests one and two. Results: The students performed better in the second test, discrete point test rather than the first which was an integrative test. The same grammatical mistakes in the composition were used correctly in the multiple choice tests by the students. Conclusion: Our findings show that student perform better in non-productive rather than productive test. Since being competent English language user is an expected outcome of university language courses it seems warranted to switch to integrative tests as a measure of English language competency. Keywords: Integrative tests, english language for medicine, Academic English

Developing an English Proficiency Test for Fourth-Year Secondary Students

The researcher-constructed English proficiency test, after having been administered and subjected to item analysis, yielded 80 valid items, or 61.54% out of 130 original items under the discrete-point test; and 49 items, or 75.38% out of 65 items under the integrative test. Those valid items constituted the final items in the revised form of the discrete-point and integrative tests. Both tests proved to have reliability coefficients of 0.84 for the discrete-point test and 0.97 for the integrative test. The use of both types of test is recommended to effectively evaluate learner's English language proficiency. English teachers must also be reoriented to the combined use of the two types of test to effectively determine students' actual English proficiency level.

Profiling Flexible Highway Pavement Continuously with Rolling Dynamic Deflectometer and at Discrete Points with Falling Weight Deflectometer

Nondestructive testing of pavements plays an important role in the management of pavement infrastructure. Nondestructive testing techniques have been widely utilized in the United States during the past decade to characterize pavement systems at discrete points. Discrete-point testing is, however, not feasible for routine use as a means of continuously characterizing pavement systems. A new nondestructive testing technique for continuously profiling pavements has been developed, namely, the rolling dynamic deflectometer (RDD). The RDD is a large truck on which a servohydraulic vibrator is mounted. The vibrator is used to apply large vertical dynamic loads to the pavement with loading rollers, and the induced dynamic displacements are measured at multiple locations with rolling sensors. This arrangement permits continuous measurements at multiple locations on the pavement surface in a fraction of the time required for typical discrete-point testing at a few points along the section. In this paper, the configuration of the RDD and the procedure used to analyze RDD data are discussed. Continuous deflection profiles are presented from a test section of a flexible Interstate highway pavement, which contains several distinctly different pavement structures. The effects of the structures on measured deflections are discussed, and the capability of the RDD to delineate the extent of each feature is presented. The importance of loading frequency and resonances in the pavement structure is shown. In addition, results from the RDD are compared with results from falling weight deflectometer (FWD) tests performed at the same locations.

Constructing rating scales for second language tests

Second language testing increasingly uses rated tasks in place of objectively scored items. Rating language performance is more demanding than scoring discrete-point tests. Lower reliability and validity for ratings are to be expected, especially in instructional settings where differences among students are small, and training for the use of rating scales is infrequent. The need for rating scales has not always led to effective and efficient scales, however. Standard rating scales require the matching of examinee performance to a verbal description. An alternative type of scale consists of an empirically derived, ordered set of binary questions relating to boundaries between levels on the performance being evaluated. Rating depends on a series of decisions. Two scales of this type were developed to assess the grammatical accuracy and communicative effectiveness of a story-retell task. All scale categories were used, thus demonstrating that category range was appropriate to the learners. Reasons are offered to explain why this type of scale may ameliorate reliability and validity problems associated with standard rating scales.

The Oral Proficiency Interview: Discrete Point Test or a Measure of Communicative Language Ability?

ABSTRACT This article1 reports the results of a study that investigated the relative contribution of discrete points of sentence level grammar and communicative factors on determination of proficiency level. Raters' responses on questionnaires concerning their choice of rating reveal that communicative factors rather than features of discrete‐point sentence level grammar were most influential in determining a subject's proficiency level. These data provide support for the ACTFL Guidelines and suggest that the concerns of some critics with respect to the Guidelines' failure to mention communicative competence are unwarranted.

Het Benul Van De Zin

Eight- to ten-year-olds no longer write the way they talk: they make a distinction between speech and writing. However, they lack specialized knowledge on written language. A key issue is their lack of knowledge on sentences. The fully-fledged sentences of written language do not come naturally to language users. If children demarcate sentences in writing, they are guided by their tacit grammatical knowledge only. This knowledge misguides them. Sentences can be considered as writers* building blocks, the smallest units with which to plan and to organize text. Results of an experimental study into the effects of grammar instruction upon writing by young children not only suggest that it is very well possible to teach junior writers what are sentences in the a-rhetorical context of grammar class; children spontaneously put to use this explicit, declarative knowledge while writing, too. Thus, it seems, knowledge transfers spontaneously. Moreover, effects in a discrete point test may indicate that instruction in the grammar of sentences has students start exploiting sentences for what they are: a scaffold for writing.

Development and Analysis of a Flexible Spanish Language Test for Placement and Outcome Assessment

Enrollment in foreign languages at B. A. -granting institutions in the United States has been increasing dramatically in recent decades' (see, for example, Brod, 1988). Recent studies and commissions have recommended more language study at all levels of schooling, and universities have offered new opportunities for students to pursue international careers through interdisciplinary studies. Increases in foreign language study have also evolved through strengthening of school and college language requirements (see Brod and ILapointe, 1989). Large state institutions may be especially vulnerable to a myriad of problems brought about by burgeoning enrollments in foreign language. Computer registration has decreased informal advising and counseling for placement. At large institutions, such as the University of Iowa, placement tests are administered to a great number of students during a one-hour period. The results are required for placement decisions within hours of the test administration. Developing and administering tests for placement and assessment of large numbers of foreign language students is a major undertaking. In addition, with increased enrollments, teacher training and testing of foreign language teachers has become an even greater responsibility. The creation of the ACTFL Oral Proficiency Interview (OPI) has offered a form of oral testing intended for the college setting, yet the OPI has been criticized as being limited in the type of discourse elicited. (See, for example, Barnwell, 1980; Valdes, 1989; Bachman and Clark, 1987; and Lantolf and Frawley, 1985). Furthermore, the OPI training demands considerable expenditure of money and time. These financial and time constraints make such oral testing feasible only at critical junctures or for borderline cases in placement. Machine-scored tests of listening comprehension and reading are promising modes in which ACTFL has made little progress to date. Tape-recorded speaking tests, writing correction tests, and dictations may also prove to be useful for placement testing in foreign languages. Tape-recorded speaking tests can be graded more efficiently than the OPI and they still collect a speech sample; writing correction tests can be done in a multiple-choice format allowing for machine-scored grading; and dictation can be quickly graded by counting each word as either right or wrong. Several institutions have refined placement procedures that are not based on seat time, for example, the University of Arizona (Schultz, 1988), Brigham Young University (Larson, 1989), the University of Minnesota (Lange, 1987), the University of Pennsylvania (Freed, 1987), the University of South Carolina (Mosher, 1989). Nevertheless, recent surveys have shown that foreign language placement testing procedures are neither systematic nor satisfactory for most institutions. (Wherritt and Cleary, 1990; Klee and Rogers, 1989). Commercially available tests have had three critical problems: availability only in predetermined difficulty levels, limitations to discrete-point testing, and the large expense associated with purchase of test booklets. This paper reports on the development and field test of an item bank developed at the University of Iowa for placement and assessment of language students.

Language disorder or difference: assessing the language skills of Hispanic students.

Results from a study on current practices of Spanish-speaking speech and language specialists are reported. Analyses of the data indicate that a heavy diagnostic emphasis continues to be placed on the results of discrete-point tests. A protocol to assess limited-English-speaking students suspected of a language or learning disability is offered.

Qualitative Analysis of Immediate Recall Protocol Data: Some Classroom Implications

A number of recent articles have favored the immediate recall protocol as a valid means of assessing reading comprehension for foreign language students. Both Johnston and Bernhardt, among others, have extolled the virtues of this instrument for evaluating reading comprehension. Johnston claims that the immediate recall procedure the most straightforward assessment of the result between text-reader interaction.' Moreover, Bernhardt asserts that this offers insight into the method of reconstruction which [the reader] employs to encode information in a text2 She also maintains that the immediate recall protocol avoids many of the pitfalls commonly found in other assessment measures. For example, it does not allow students to guess their way through the text (a pitfall of the multiple-choice testing format) nor does it influence students' understanding of the text.3 In short, the immediate recall protocol demands that the reader comprehend the text well enough to be able to recall it in a coherent and logical manner. More importantly, this procedure allows misunderstandings or gaps in comprehension to surface; a feature that other methods of evaluation cannot offer. Unlike the recall protocol, discrete point testing cannot capture students' interactive understanding of the text, which is now considered to be the true nature of reading comprehension. Having acknowledged that the recall protocol is clearly a superior way to assess reading comprehension, this still leaves foreign language teachers with the problem of how to score the recalls. Much progress has already been made in this area with regard to quantitative scoring methods. For instance, Bernhardt proposed that the students' score on the recall equal the number of the idea units that they remember from the texts.4 Similarly, Brozovic devised a scoring system for literary texts based on Golden's of separating passages into episodes.5 The score obtained using Brozovic's system reflects the total number of episodes that the student recalls.

Comments on Yael Bejarano's "A Cooperative Small-Group Methodology in the Language Classroom". A Reader Reacts

1. One of the three achievement tests used in the study, according to the recent report, was a [italics added] grammar and vocabulary (p. 490). The discrete-point test was actually a combination of two separate sections, according to the 1984 report. One section consisted of 5 items that required the formulation of interrogative sentences, and the other section included a 10-item cloze test.