Multiple-choice Test Items Research Articles

An Evaluation of 4PL IRT and DINA Models for Estimating Pseudo-Guessing and Slipping Parameters

In an achievement test, the examinees with the required knowledge and skill on a test item are expected to answer the item correctly while the examinees with a lack of necessary information on the item are expected to give an incorrect answer. However, an examinee can give a correct answer to the multiple-choice test items through guessing or sometimes give an incorrect response to an easy item due to anxiety or carelessness. Either case may cause a bias estimation of examinee abilities and item parameters. 4PL IRT model and the DINA model can be used to mitigate these negative impacts on the parameter estimations. The current simulation study aims to compare the estimated pseudo-guessing and slipping parameters from the 4PL IRT model and the DINA model under several study conditions. The DINA model was used to simulate the datasets in the study. The study results showed that the bias of the estimated slipping and guessing parameters from both 4PL IRT and DINA models were reasonably small in general although the estimated slipping and guessing parameters were more biased when datasets were analyzed through the 4PL IRT model rather than the DINA model (i.e., the average bias for both guessing and slipping parameters = .00 from DINA model, but .08 from 4PL IRT model). Accordingly, both 4PL IRT and DINA models can be considered for analyzing the datasets contaminated with guessing and slipping effects.

The Pedagogical Content Knowledge of the School of Education Pre-Service Teachers in Test Construction

Pre-service teachers, as part of their training, are called to demonstrate skills in selecting, developing, applying, using, communicating, and evaluating their future students’ assessment information and assessment practices. The study sought to determine the pedagogical content knowledge of pre-service teachers in test construction. The study employed a descriptive research design and was conducted in the School of Education at Xavier University. The sampling method used is simple random sampling and made use of Cochran’s formula of estimation for getting the sample size. The results showed that the pre-service teachers have “Fair” pedagogical knowledge in the essentials of test construction and need to improve in writing instructional objectives and constructing the table of specifications. However, they have “Good” pedagogical knowledge in the majority of the objective-type tests, except in the construction of completion and multiple-choice test items. The results also showed that there is no significant difference between the respondents’ pedagogical content knowledge in the preliminaries or the essentials of test construction when grouped according to their undergraduate program. Also, there is no significant difference with the respondents’ pedagogical content knowledge in the construction of objective-type tests when grouped according to their Undergraduate Program.

Development of the Adult ECMO Specialist Certification Examination

The American Society of Extracorporeal Technology Board of Directors, consistent with the American Society of Extracorporeal Technology’s safe patient care improvement mission, charged the International Board of Blood Management to write a knowledge and skill certification examination for healthcare personnel employed as adult extracorporeal membrane oxygenation (ECMO) specialists. Nineteen nationally recognized ECMO subject-matter experts were selected to complete the examination development. A job analysis was performed, yielding a job description and examination plan focused on 16 job categories. Multiple-choice test items were created and validated. Qualified ECMO specialists were identified to complete a pilot examination and both pre- and post-examination surveys. The examination item difficulty and candidate performance were ranked and matched using Rasch methodology. Candidates’ examination scores were compared with their profession, training, and experience as ECMO specialists. The 120-item pilot examination form ranked 76 ECMO specialist candidates consistent with their licensure, ECMO training, and clinical experience. Forty-three registered nurses, 28 registered respiratory therapists, four certified clinical perfusionists, and one physician assistant completed the pilot examination process. Rasch statistics revealed examination reliability coefficients of .83 for candidates and .88 for test items. Candidates ranked the appropriateness for examination items consistent with the item content, difficulty, and their personal examination score. The pilot examination pass rate was 80%. The completed examination product scheduled for enrollment in March 2020 includes 100 verified test items with an expected pass rate of 84% at a cut score of 67%. The online certification examination based on a verified job analysis provides an extramural assessment that ranks minimally prepared ECMO specialists’ knowledge, skills, and abilities (KSA) consistent with safe ECMO patient care and circuit management. It is anticipated that ECMO facilities and ECMO service providers will incorporate the certification examination as part of their process improvement, safety, and quality assurance plans.

Dimensionality assessment of binary response test items: A non-parametric approach of bayesian item response theory measurement

This study investigated dimensionality of Binary Response Items through a non-parametric technique of Item Response Theory measurement framework. The study used causal comparative research type of non-experimental design. The sample consisted of 5,076 public senior secondary school examinees (SSS3) between the age of 14-16 years from 45 schools, which were drawn randomly from three senatorial districts of Osun State, Nigeria. Instrument used for this study was 2018 Osun State unified multiple-choice mathematics achievement test items with empirical reliability coefficient of 0.82. Data obtained were analysed using Non-linear factor analysis, Stout’s Test of Essential Unidimensionality (STEU), Factor Analysis (FA), Full Information Factor Analysis (FIFA) and Bootstrap Modified Parallel Analysis Test (BMPAT). Results showed that both the BMPAT and STEU ascertained violation of unidimensionality assumption of the test items (the observed difference in the second eigenvalue of the observed data and that of second eigenvalue of the simulated data was statistically significant, p = 0.0099; Stout’s test rejected the assumption of essential unidimensionality, T = 10.6260, p<0.05). Non-linear factor analysis and full information factor analysis revealed that four dimensions embedded in the test items and loadings of the items showed within-item multidimensionality respectively. The authors’ concluded that modeling examinees’ performance with unidimensional model when it was actually multidimensional in nature would affect performance of examinees adversely and could lead to blur judgment. Consequently, it is recommended that unidimensional scoring method of Osun State unified mathematics achievement test implicit in Classical Test Theory should be jettisoned and an appropriate scoring model (multidimensional) should be embraced.

Permutation of UTME Multiple-Choice Test Items on Performance in Use of English and Mathematics Among Prospective Higher Education Students

In an attempt to curtail examination malpractice, the Joint Admission and Matriculation Board (JAMB) has been generating different paper types with a different order of test items in the Unified Tertiary Matriculation Examination (UTME). However, the permutation of test items may compromise students’ performance unintentionally because constructive suggestions in theory and practice recommend that test items be sequenced in ascending order of difficulty. This study used data collected from a random sample of 1,226 SSIII students to ascertain whether the permutation of test items has any effect on the performance of students in two different subjects (Use of English and Mathematics). The study adopted the Equivalent Groups Quasi-Experimental Research Design with three independent groups. Findings emerged, amongst others, that there is a significant difference in the performance scores of prospective university students’ in use of English and Mathematics examintaions arranged in three different orders (ED, DE, R). There are no significant gender differences in the performance of students in Use of English and Mathematics based on test item permutation. However female students perform better than male students when test items are arranged in ascending order of difficulty while males perform better when test items are arranged in descending order of difficulty. It was concluded that the permutation of test items in UTME examination tends to affect the performance of students in Use of English and Mathematics. This finding has implications for the future conduct of UTME examinations and enrolment into higher education as the randomization of UTME test items changes the difficulty order of different paper types. It was recommended that other measures of curtailing examination malpractices that would not affect students’ academic performance should be adopted.

Psychometric Analysis of Economics Achievement Test Using Item Response Theory

This study determined the psychometric properties of the Economics Achievement Test (EAT) using Item Response Theory (IRT). Two popular IRT models namely, one-parameter logistics (1PL) and two-parameter logistics (2PL) models were utilized. The researcher adopted instrumentation research design. Four research questions and two hypotheses were formulated to guide the study. The population size is five thousand, three hundred and sixty-two (5,362) from thirty-seven (37) schools. The sample for the study was 1,180 senior secondary school students (SSS3) drawn using multi-stage sampling procedure. The 1,180 students were stratified according to gender which resulted to 885 females and 295 males. The instrument for the study consisted of 50 multiple-choice test items on the economics achievement test, developed by the researchers. Reliability and validity for each item and for the whole test were established according to the one-parameter and two-parameter logistic models. Research question one was answered using 1PLM, while research questions two and three were answered using 2PLM IRT model. Hypothesis one was tested using t-test analysis of difference between the difficulty parameters estimated using 1PLM and 2PLM while hypothesis two was tested using Chi-square. The finding of the study revealed significant difference between the item difficulties estimated using 1PLM and 2PLM. Also the observed scores of the testees on the test items fit the 1PL 2PL models.

The Impact of High School Life Science Teachers' Subject Matter Knowledge and Knowledge of Student Misconceptions on Students' Learning.

One of the foundational assumptions in education is that greater teacher knowledge contributes to greater gains in student knowledge, but empirical evidence in support of this assumption is scarce. Using a U.S. sample of 79 biology teachers and their 2749 high school students, we investigate whether teachers’ subject matter knowledge (SMK) and knowledge of students’ misconceptions (KOSM) in high school life science are associated with students’ posttest performance on multiple-choice test items designed to reveal student misconceptions, after controlling for their pretest scores. We found that students were more likely to answer an item on the posttest correctly if their teachers could answer the question correctly, themselves (SMK). Teachers’ ability to predict students’ most common wrong answer (KOSM) for an item predicted even better student performance. Items for which a particular wrong answer rose above others in popularity saw an even greater benefit for teacher KOSM.

English

This study examined the effectiveness of computer–assisted and excursion strategies on secondary school students’ achievement in Biology towards sustainable development. The study adopted quazi experimental research design. The population consisted of all the biology students in public senior secondary schools in South West, Nigeria. The sample for the study consisted of 240 senior secondary year two (SS2) Biology students selected from six schools in the three states from South West Nigeria using multistage sampling technique. The subjects were divided into the two treatments and control groups, that is, Computer Assisted, Excursion and control groups. A forty item multiple choice test on Biology was administered to the students after exposing the experimental groups. The treatment is teaching. Three hypotheses were formulated and tested; the data generated were analyzed using t-test and Analysis of Covariance (ANCOVA). The result showed that the achievement of students in the two experimental groups were better than those in the control group. Students taught with excursion strategy achieved better than those taught with Computer Assisted strategy. Also, location has significant effect on achievement while gender has no significant effect on achievement in Biology. Based on the discoveries, it was recommended that Excursion strategies and Computer – Assisted should be employed in the teaching of Biology as innovative strategies. Key words: Science students, teaching strategy, research, sustainable development.

Targeted Linguistic Simplification of Science Test Items for English Learners

In this experimental study, 20 multiple-choice test items from the Massachusetts Grade 5 science test were linguistically simplified, and original and simplified test items were administered to 310 English learners (ELs) and 1,580 non-ELs in four Massachusetts school districts. This study tested the hypothesis that specific linguistic features of test items contributed to construct-irrelevant variance in science test scores of ELs. Simplifications targeted specific linguistic features, to identify those features with the largest impacts on ELs’ test performance. Of all the linguistic simplifications used in this study, adding visual representations to answer choices had the largest positive effect on ELs’ performance. These findings have significant implications for the design of multiple-choice test items that are fair and valid for ELs.

Measuring Nature of Science Views of Middle School Students

Developing scientific literacy for all students is the most often stated purpose of contemporary science education. Nature of science (NOS) is seen as an important component of scientific literacy. There are various perceptions of NOS in the science education community and NOS itself is an ever-changing construct. This makes it challenging to develop instruments for measuring understanding of NOS at different levels. Many instruments have been developed and are being developed to assess NOS learning, which indicates the importance attributed to this subject. In this study, we developed a multiple-choice test to measure NOS understanding of middle school students. The instrument was applied to 1397 middle school students. The 24 item multiple-choice test had KR-20 reliability coefficient of 0.74. A 12 item multiple-choice test created as a subset of the 24 items of the original test. This test was easier and had higher discrimination, which can provide useful measurement data about students’ understanding of NOS for diagnostic or formative purposes.

Development of three-tier diagnostic test instruments for detecting students’ conception

A research and developmental study about developing three tier test instrument was carried out to detect students’ conception on cell concept. Students’ pre-conception about cell concept, cell reproduction and gametogenesis are often not in line with scientific concepts. This preliminary study was conducted to analyze the profile of students’ pre-conception about cell reproduction in senior high schools. Modified Borg and Gall’s R&D research design used in this research. Instrument development resulted in three tier diagnostic test in the form of 30 multiple choice test items with its reason and certainty index. To validate the instrument, it was administered towards a number of senior high school students (XI and XII graders) in Bandung. Then the response of the students were quantitatively and qualitatively analyzed. Research findings show that the instrument as product is effective to determine students’ conception, including their misconception. The percentage of misconception found in cell concepts mostly in gametogenesis (17.35%), compared to those who comprehend (27.83%), and not comprehend (54.82%). The results of this preliminary study will then be used for further study in preparing STEM based learning program which will facilitate 12th grade students on conceptual change in cell reproduction, as well as to prevent its misconception.

The Analysis of Reading Exercise in English Textbook Entitled Pathway to English for The SENIOR High School Grade X

This study is aimed to analyze the types of Reading Exercises on the English textbook used by the first year students of SMA SWASTA METHODIST 7 Medan. The objective of the study is to find out the types of reading exercises on English textbook used by the first year students of SMA SWASTA METHODIST 7 Medan. The design of the study is descriptive qualitative research. The qualitative data were obtained by the steps that mentioned in the procedure of the research, i.e. reading, identification, classification and simplification. The researcher analyzed the exercise in three steps,identifying the topic, clutser the topic, draw explanation. The object of the study is the reading exercises created by Th. M. Sudarwati and Eudia Grace entitled by Pathway To English and published by Erlangga, 2017. The data was collected only by taking documentary analysis. It means that the writer document reading exercises on students� English textbook and analyzed its types of reading exercises. The final result of this study shows that are five types of reading exercises on the English textbook they are Types of reading exercise are Matching Test, True/ False Reading Test, Multiple Choice item test, Completion item tests, and Long and Short answer questions. The result of reading exercise analysis shows that there are controlled exercise and guided exercise. In controlled exercises the researcher didn�t find exercise in Pathway to English textbook. And in guided exercises there are find in multiple choice in vocabulary 2 exercise. Matching cued word matching 3 exercise, matching picture cued sentence 1 exercise, vocabulary matching 5 exercise, matching selected response fill in vocabulary 7 exercise. True or false 3 exercise, completion item in the following of text 1 exercise, completion are in the text itself 9 exercise. In short answer question 6 exercise, long answer question 8 exercise. But, the researcher didn�t find exercise in multiple choice contextualized vocabulary / grammar and multiple choice vocabulary / grammar.

AN ANALYSIS OF RELATIVE CLAUSES MADE BY THE EIGHT GRADE STUDENTS OF SMP NEGERI 3 SELAT IN ACADEMIC YEAR 2017/2018

The present study is dealt with An Analysis of Relative Clauses Made by the Eight Grade Students of SMP Negeri 3 Selat in Academic Year 2017/2018. The researcher was interested in making an analysis of Relative clauses made by the students under study. The present study is intended to assess the acquisition of relative clauses made by the eighth-grade students of SMP N 3 Selat Karangasem in academic year 2017/2018. In other words, this study is to figure out the ability of the subject under study in understanding relative clauses. The present’s study which use ex-post facto research design with descriptive analysis was intended to answer the following research question: to what extent is the acquisition of relative clauses made by the eighth-grade students of SMP N 3 Selat Karangasem in academic year 2017/2019? In order to answer the research question before, the data were required for the present study which in the form of raw score is showing the ability of the students in mastering relative clauses, were collected by administering the valid and reliable research instrument in form of multiple-choice test items. Then the data is analyzed by using Norm-Reference measure with five standard values. The populations of the present study are the eighth-grade students of SMP Negeri 3 Selat which consist of 206 students. They are considered to have heterogeneous characteristics. In this study 30 students are considered as a sample of the study by using random sampling technique. The data of the study is clearly showed that: (1) there are 4 students or (13.33%) out of 30 students under study who showed excellent ability of relative clauses. (2) There are 4 students or (13.33%) out of 30 students under study who showed good ability of relative clauses (3) there are 10 students or (33.33%) out of 30 students under study who showed sufficient ability of relative clauses. (4) There are 12 students or (40%) under 30 students under study who showed insufficient ability of relative clause. (5) No one students or (0%) out of 30 students who showed poor ability of relative clauses. In the result of the study were used as the basses to assign level/grade to the subject under study, there are 12 students or 40% out of 30 students under investigation who fail in learning and acquiring relative clauses. It means that 40% of them still have low ability. The finding of the present study can be used as the basses in planning much better language teaching especially in mastering grammar or relative clauses.

THE DEVELOPMENT OF STUDENTS’ MATHEMATICAL LITERACY PROFICIENCY

Improving the proficiency in mathematical literacy is one of the efforts to improve the human resources quality. Its development describes the students’ strengths and weaknesses to support the improvement. The research tried to describe the development of students’ proficiency in mathematical literacy, based on the school level and the students’ gender. The research was a cross-sectional developmental study. The population was all of students ranging between 13 and 16 years old in the Province of Yogyakarta Special Region, Indonesia. In the sample were 1,001 students in the 8th, 9th and 10th grade, determined to use the combination of stratified and the cluster random sampling technique. The data were collected using 30 items. The data analysis was conducted by estimating students’ ability through the item-response theory approach under the Rasch model and by using the parameters that had been adjusted to the international study item parameter under the concordance model for the linking score to interpret the students’ development. The research results showed that there has been improvement of literacy proficiency among the 8th, 9th and 10th grade students. The development would generate higher scores utilizing multiple choice test items. There was a tendency that the school level, gender, the form of items, content, context, and process literacy would influence the mathematical literacy score. The efforts for improving proficiency in mathematical literacy were discussed. Keywords: ability development, mathematical literacy, PISA released items, cross-sectional developmental study

Development and Validation of Multiple-Choice Test of the Geometry Part of Mathematics for Secondary Class

This study overviews development and validity of a pool of multiple choice test items of geometry part of Mathematics for secondary level. A table of specifications was prepared and a pool of 48 multiple-choice type test items was developed from the test universe. The content validity and face validity of test items was determined with the help of a team of experts. Sample of the study was 488 students of class 10. After determining the validity and reliability through item analysis and quantitative and qualitative analysis of the test, it was concluded that 30 out of 48 items in the test were valid, reliable and suitable for measurement of the learning achievements in the course. Therefore, these items are useful in the boards of examinations for 9th class in Punjab (Pakistan).

Methods used by mathematics teachers in developing parallel multiple-choice test items in school

The study was aimed at describing five methods of the development of parallel test items of the multiple-choice type in mathematics at Yogyakarta (primary education level). The study was descriptive research involving 22 mathematics teachers as the respondents. Data collection was conducted through interviews and document reviews concerning the developed test packages. A questionnaire was used to gather data about the procedure the teachers employed in developing the tests. Findings show that the teachers used five methods in developing the test item; namely (1) randomizing the item numbers; (2) randomizing the sequences of response options; (3) writing items using the same contexts but different figures; (4) using anchor items; and (5) writing different items based on the same specification table. All of the respondents stated that they developed the table of the specification before developing the test items and that most of them (77%) did the validation of the instruments in content and language.

Effect of Biology Practical Activities on the Academic Achievement of Senior Secondary School Biology Students.

This study investigated the effect of Biology practical activities on the academic achievements of senior secondary school biology students in Awgu Local Government Area of Enugu State. The study adopted pretest posttest non-randomized quasi experimental design. Four research questions and two hypotheses guided the study. One hundred and twenty (120) senior secondary school biology students were sampled from the population of 490 SS II students. Four schools were sampled out of 13 Government Secondary Schools in Awgu L.G.A. Instrument for data collection was Biology Achievement Test (BAT) consisting of 21 multiplechoice test items was constructed. The instrument was validated by three experts; one in measurement and evaluation and two in biology education. The reliability test was done using Kuder-Richardson (K-R20) which gave a reliability index of 0.74. The research questions were answered using mean and standard deviation while the hypotheses were tested using t-test. The findings of the study showed that students taught biology using practical activities performed better than their counterparts using conventional method. The result also showed that gender has significant difference on students‟ performance when taught with practicals. The researchers therefore recommended that teachers should employ practical activities in teaching biology to enhance students‟ performance.

L1 to L2 Translation Practices in Foreign Language Teacher Education: Student Teachers’ Views

The purpose of this study was to elicit student teachers’ views regarding the translation practices they were engaged in within the content of a compulsory ‘L1 to L2 translation’ course. During a 12-week translation class in 2018 Spring semester, 22 student teachers were asked to translate different types of texts: literary texts, recipes, magazine articles, official documents, and oral texts. They also deal with different translation practices such as prepared translation, unprepared in-class translation with and without dictionaries or online translation tools, constructing multiple-choice translation test items, and oral translation. At the end of the training, the student teachers were asked to rank those different practices and text types according to their usefulness and difficulty. They were also asked to explain their reasons for those ideas. The quantitative and qualitative analyses of the obtained data showed that each translation practice and each text type had its advantages and disadvantages for student teachers in their way to become foreign language teachers. A number of suggestions and implications are also provided for student teachers and teacher trainers for the translation courses in teacher education programs.

ASSESSMENT OF MODEL FIT FOR 2016 AND 2017 BIOLOGY MULTIPLE CHOICE TEST ITEMS OF THE NATIONAL BUSINESS AND TECHNICAL EXAMINATION BOARD

This study was based on the assessment of model fit for 2016 and 2017 Biology multiple choice test items of the National Business and Technical Examination Board. It aimed at empirically investigating the model fit of the 1, 2, and 3 Parameter Logistic Models (PLM) of the examinations using Item Response Theory. Three research questions were raised with two hypotheses formulated and tested. The expo-facto research design was adopted for this study. The population for the study was 5,115 and 4600 candidates in public and private schools in south-south geo-political zone in Nigeria for 2016 and 2017 respectively. A total of 2000 students were sampled using Simple random sampling technique. The instruments for data collection was the NABTEB 2016 and 2017 Biology multiple choice question papers. The instruments are said to be valid and reliable as they were developed by a standard examination body. The responses from the instruments were used for data analysis. The results obtained from the study revealed that the 1, 2 and 3 PLM fit the 2017 and 2016 NABTEB May/June Biology multiple choice test items. However, the 1PLM provided a better fit to the data than other models. Based on the findings of the study, it was recommended among others that the examining bodies should make sure that model fit the data well before they are used to make inferences regarding the data.

Psycho productive skills multiple choice test items for assessing students in mechanical engineering craft in technical colleges in Nasarawa State, Nigeria

This study focused on the development and validation of psycho productive skill multiple choice test (PSMCT) for assessment of technical college students’ achievement in mechanical engineering craft. The study adopted the instrumentation research design and was conducted in Nasarawa State, Nigeria. The population of the study was 248 National Technical Certificate (NTC) III students comprising three capability groups (42 high, 140 average and 66 of low capability). A 305 item draft copy of psycho productive skills multiple choice test items was generated and utilized as the instrument for the study. The test was used to assess students in the four Government Technical Colleges in Assakio, Mada Station, Agwada, and Doma in the study area. The data collected were analyzed using split-half technique and Kadder-Richardson (K-R20). The reliability coefficient of the test was 0.84. It was found out that there were significant differences in the mean scores of the three ability groups. It was recommended among others, that examination bodies should integrate the psycho productive skill multiple choice test in their examination process for certification of NTC students. Key words: Mechanical engineering craft, assessment, psycho productive skills multiple choice test, technical college, students, Nasarawa State.