Multiple-choice test items with “none of the above” as an alternative

The present study was initiated to investigate the comparability of multiple-choice and true-false item formats when the time necessary to respond to each type of item was equated empirically. Also investigated was the relative difficulty of multiple-choice (MC), true true-false (Tf), and false true-false (tF) items mea suring the same content. Results indicated that true-false items result in a less reliable test than one using a four-option MC format, even when empirically determined differences in time needed to answer the respective formats were taken into account. When scores were corrected for guessing, the MC items were significantly easier than the true-false format. CONSIDERABLE DISCUSSION has taken place among measurement specialists regarding the virtues of multiple-choice versus true-false test item for mats. Recent contrasting examples might include .. .the advantages attributed to (true-false) are not, unfortunately, very valid... (3:160), and .. .a few (test specialists) see special virtues of efficiency and ease of preparation in (true-false items) and advocate their wide use (2:1) The most obvious limitation of true-false relative to multiple-choice test items is the degree to which the former is subject to guessing. Several studies have shown that the reliability of a test is directly related to the number of choices per item (1, 4, 5, 6) . Similarly, it would be expected that a multiple choice test would have greater reliability than a true false test if the number of items were held constant. However, since a greater number of true-false items can be administered per unit time, it is possible that in a given amount of time, the increased number of true-false items administered would allow for greater reliability and more efficient sampling of content objectives than had a multiple-choice format been used. Using eighty-eight multiple-choice items from a published test in natural science, Ebel (2) compared formats by rewriting each multiple-choice item as a parallel true-false item. Two forms, each con sisting of forty-four multiple-choice and forty-four true-false items, were developed. Reliabilities (K.R. 20) were computed for the multiple-choice and true-false sections of both forms, and assuming that true-false items could be answered per multiple choice item, the Spearman-Brown formula was used to predict the reliability of an 88-item true false test. For the first form, this adjusted relia bility was greater than the reliability obtained for the multiple-choice section of the test; however, the inverse was true with respect to the second form. The present study was concerned with several currently unanswered questions. First, what is an empirically determined ratio of multiple-choice to equivalent true-false items which can be answered in a given amount of time? Second, for achieve ment test items administered within a classroom situation, which of the two formats under consider ation result in greater reliability per unit of testing time? Third, what is the relative reliability of true true-false and false true-false items when compared to multiple-choice items? Fourth, what ratio of multiple-choice to equivalent true-false items is necessary for producing equal reliability coefficients? Lastly, after equating for differences in the effect of guessing, what is the relative difficulty of the different formats?

The purposes of this study were to (a) Look at what teachers are being told about multiple-choice test item construction by introductory educational psychology textbook authors and (b) to evaluate the quality of test items that preservice teachers may be exposed to if their university instructors use unedited multiple-choice items provided by the publishers of their classroom text. A comprehensive survey of educational psychology texts was completed to identify the textbook authors’ recommended guidelines for teachers to follow when writing multiple-choice test items. Next, a reduced set of 12 guidelines was identified on the basis of the frequency of author recommendations. These 12 guidelines were used to evaluate 60 multiple-choice items (N = 1,080) that were randomly selected from 18 different instructor guides to introductory educational psychology texts. The results indicated that approximately 60% of the items violated one or more guidelines.

The purpose of this study was to evaluate the effectiveness of multiple-choice questions in assessing the clinical judgment abilities of prelicensure registered nursing students. Nurse educators are responsible for ensuring fairness in testing and evaluating the development of clinical judgment abilities of students throughout the nursing program of study. This study compared multiple-choice test items developed using the current NCSBN® NCLEX-RN test item writing guidelines to multiple-choice test items developed using the new guidelines proposed by the NCSBN® Next Gen NCLEX-RN® project. The results indicated a statistically significant difference between the two types of multiple-choice items in the measurement of clinical judgment. Participants answered all the items developed according to the current NCSBN® NCLEX-RN test plan correctly within each of the five physiological categories 41% of the time, as compared to only 14% of the time on the items written according to the recommendations of the NCSBN® Nursing Clinical Judgment Model in the Next Gen Project. These findings indicate the participants understood the theoretical knowledge of the physiological concepts but could not apply the theoretical knowledge to the clinical scenario presented in the multiple-choice questions created using the NCSBN® Nursing Clinical Judgment framework. The information gained from this study provides evidence of a process that nurse educators can follow using the NCSBN® Nursing Clinical Judgment model as a guide to create multiple-choice test items to assess the clinical judgment level of nursing students.

This study compares two-option multiple-choice vocabulary test items with comparably written true-false test items. Results indicate that the multiple-choice format provides a significantly more reliable measure than the true-false format. Results further indicate that the true and false items do not appear to be measuring the same abilities. Potential strategies are discussed that an examinee may use for solving two-option multiple-choice and true-false test items in an effort to explain these results.

The aim of the current study was to identify the impact of the number of distractors in multiple-choice test items on the psychometric characteristics of the test items and the item information function according to the two-parameter logistic model in the Item Response Theory (IRT). To answer the study questions, an achievement test was built covering all school tests course that measures teachers' degree of knowledge to build school tests. The test consisted of (42) items and was of 3 different forms varying in the number of options. The study sample consisted of 356 male and female teachers from Ma'an governorate and Wadi al-Sair district and was selected in a simple random manner. The statistical program (SPSS) and the (Bilog-mg3) program were used to analyse the responses of the study sample. The results of the study showed that there were statistically significant differences between the means of the values of the test items information function in favor of the four-distractors test form compared to the three-and-five- distractors test form. The results, furthermore, showed that there were statistically significant differences between the means of the values of the test items information function in favor of the five-distractors test form compared to the three-distractors test form.The results, further, showed that there were no differences between the means of the estimate of the items difficulty parameter due to the number of distractors of the item. It was indicated that there were differences between the means of the estimate of the items discrimination parameter concerning the three-and-four-distractors tests in favor of the four-distractors test. It was, besides, shown that there were statistically significant differences between the means of the estimate of the items discrimination parameter concerning the three-distractors and five- distractors tests’ forms in favor of the for the five- distractors test. Keywords: number of distractors, multiple-choice test, item response theory, items’ parameters, information function DOI: 10.7176/JEP/13-13-07 Publication date: May 31 st 2022

صعوبة الفقرة | التصحيح لأثر التخمين | المخاطرة في الإجابة | فقرات الإختيار من متعدد، المعرفة الجزئية | معادلة التصحيح | Item Difficulty | Correction for Guessing | Risk - Taking | Multiple Choice Items | Partial Knowledge | Scoring Formula

Grading is often a faculty member’s least favorite chore, especially in engineering where open-ended problems prevail. For this reason, multiple-choice test items could be a popular alternative for assessing learning and understanding. In addition, most Learning Management Systems allow the instructor to cre¬ate multiple-choice questions to be scored automatically by the system. The use of multiple-choice items in engineering graphics education could increase efficiency, allowing instructors to focus on other aspects of their teaching rather than spending significant time grading open-ended problems. The authors of this paper have been involved in a project to develop a Concept Inventory for Engineering Graphics over the past several years. Since Concept Inventories typically consist of multiple-choice items, development of this instrument was reliant on the creation of numerous valid and reliable items. This paper will focus on the process employed in multiple-choice item creation with application to engineering graphics. The process will be illustrated through demonstration of item evolution through several iterations.

This quasi-experimental study aimed to compare the effect of different modes and item types of multiple-choice (MC) test items on advanced EFL learners’ listening comprehension and perception. To this end, 80 advanced EFL learners, aging 18 to 30, were selected. The participants took a listening test including dialogue-completion and question and answer multiple-choice items presented in written and oral modes. In addition, the participants were given a questionnaire on their perceptions of the oral and written modes. The results of two-way repeated measures ANOVA showed that there was no significant difference between the participants’ scores in the oral and written modes. Moreover, they received similar scores on the two item types. However, in their questionnaire responses, most of the participants preferred the written mode to the oral mode. The results also imply that although presenting the multiple-choice test items in the oral mode may form a pure test of listening comprehension, there are acceptable reasons for presenting the multiple-choice listening items in the written mode.

The study focused on development and determination of psychometric properties of multiple-choice test items for students in history in secondary schools. Two research questions and one hypothesis were involved in the study It utilized the instrumentation research design and was carried out in Nasarawa State, Nigeria. The population for the study was 248 SSIII history students using three ability groups (36 high, 70 average and 42 low abilities). There was no sampling because the population was manageable. Sixty (60) multiple-choice test items were developed from the literature reviewed and WAEC history curriculum and used in the study. The test was validated by three experts, two in the Department of History Education, and one from the Department of Measurement and Evaluation, all in the University of Nigeria, Nsukka. The test was used in assessing students in the four secondary schools in the study area. The reliability of the items was determined using test-retest method and Kudder-Richardson (K-R21). This yielded co-efficient of .86 depicting that the instrument is reliable. The Analysis of variance (ANOVA) was utilized to test the hypothesis at 0.05 level of significant. It was found out that the items had difficulty indices of between 0.30 and 0.70. It was therefore recommended that the examination bodies (WAEC and NECO) should adopt the psychometric multiple-choice test items in their examination for certification of students at the secondary school level.

Faculties are concerned about measurement of critical thinking especially since the National League for Nursing Accrediting Commission cited such measurement as a requirement for accreditation (NLNAC, 1997). Some writers and researchers (Alfaro-LeFevre, 1995; Blat, 1989; McPeck, 1981, 1990) describe the need to measure critical thinking within the context of a specific discipline. Based on McPeck's position that critical thinking is discipline-specific, guidelines for developing multiple-choice test items as a means of measuring critical thinking within the discipline of nursing are discussed. Specifically, criteria described by Morrison, Smith, and Britt (1996) for writing critical-thinking multiple-choice test items are reviewed and explained for promoting and measuring critical thinking.

This study illustrates how eye-tracking data can be translated to “item process data” for multiple-choice test items to study the relationship between subjects’ item responses and choice reading behavior. Several modes of analysis were used to test the hypothesized added value of using process data to identify choice reading patterns of subjects. In addition to the cross-sectional analises of agggregate measurements derived from the time series eye tracking data, Latent Growth Curve Model Analises were undertaken to test if the the shape of change observed in the sequential choice reading patterns differed for subjects depending on their responses to the item being correct or incorrect. Application data were from an experimental study and included seventy-one subjects’ responses to two multiple-choice test items measuring reading comprehension ability in English as a second language. Analyses were carried out for one item at a time. For each item, first, each subject’s recorded eye movements were coded into a set of Area of Interests (AOIs), segmenting the lines in the stem and the individual choices. Next, each subject’s fixation times on the AOIs were time stamped into seconds, indicating when and in what order each subject’s gaze had fixated on each AOI until a choice was marked as the correct answer, which ended the item encounter. A set of nested Latent Growth Curve models were considered for the choice-related AOIs to deliniate if distinct choice-process sequences were evident for correct and incorrect respoders. Model fit indices, random intercepts, slopes, and residuals were computed using the mean log fixation times over item encounter time. The results show that the LGM with the best model fit indicies, for both items, was the quadratic model using response variable as a covariate. Albeit limited due to the two-item – seventy-one subjects experimental setting of the study, the findings are promising and show that utilizing item-level process data can be very useful for defining distinct choice processing (task-oriented reading) patterns of examinees. Over all, the results warrant further study of choice derived AOIs using longitudinal statistical models. It is argued that, the screening methodology desribed in this study can be a useful tool to investigate speededness, distractor functioning, or even to flag subjects with irregular choice processing behavior, such as providing a direct mark on a choice, without any significant reading activity on any of the choices presented (i.e., whether cheating might have occurred.)

Writing a high-quality, multiple-choice test item is a complex process. Creating plausible but incorrect options for each item poses significant challenges for the content specialist because this task is often undertaken without implementing a systematic method. In the current study, we describe and demonstrate a systematic method for creating plausible but incorrect options, also called distractors, based on students’ misconceptions. These misconceptions are extracted from the labeled written responses. One thousand five hundred and fifteen written responses from an existing constructed-response item in Biology from Grade 10 students were used to demonstrate the method. Using a topic modeling procedure commonly used with machine learning and natural language processing called latent dirichlet allocation, 22 plausible misconceptions from students’ written responses were identified and used to produce a list of plausible distractors based on students’ responses. These distractors, in turn, were used as part of new multiple-choice items. Implications for item development are discussed.

Best Practices for Writing Test Items

(1993). C102. Second attempts at multiple-choice test items. Journal of Statistical Computation and Simulation: Vol. 47, No. 1-2, pp. 108-112.

Poor academic performance of students in senior secondary school certificate examination in some commercial subjects may be attributed to ineffective scoring techniques used by classroom teachers and examination bodies. Therefore, this study compared the effectiveness of four scoring techniques in multiple-choice financial accounting in secondary schools of Osun state, Nigeria. The study adopted a descriptive survey and Quasi-experimental research designs. The sample consisted of 420 senior secondary schools (SSSII) Financial Accounting students selected using Multi-stage sampling technique. An instrument titled “Financial Accounting Multiple-Choice Test (FAMT)” with a reliability coefficient of .80 was administered to selected students. The marks obtained through standard conventional, logical weight, corrected and confidence scoring techniques were analyzed using descriptive and inferential statistics. Findings showed that the logical weight scoring technique had the highest mean value of 40.63 when compared with other scoring techniques. It was also revealed that there was a significant difference in the performance of students in Financial Accounting multiple-choice test items using the four scoring techniques (F(3,1676) =31.494, p<.05). Based on the findings, it was concluded that students performed better using the logical weight scoring technique in Financial Accounting multiple-choice test items and is the most effective scoring technique among others. The study recommended that a logical weight scoring technique should be encouraged and used in scoring Financial Accounting multiple-choice test items in secondary schools by examination bodies.
 
 Received: 27 March 2021 / Accepted: 29 April 2021 / Published: 17 May 2021