Types Of Test Items Research Articles

Assessment Practices of Learning Outcomes of Postgraduate Students in Biomedical and Pharmaceutical Sciences at College of Health Sciences at Addis Ababa University: Student and Faculty Perspectives.

Higher education institutions are under increasing pressure to respond to societal needs which has in turn led to changes in the type of knowledge, competencies, and skills required from learners. Assessment of student learning outcomes is the most powerful educational tool for guiding effective learning. In Ethiopia, studies are scarce on assessment practices of learning outcomes of postgraduate students in biomedical and pharmaceutical sciences. This study investigated the assessment practices of learning outcomes of postgraduate students pursuing studies in biomedical and pharmaceutical sciences at the College of Health Sciences of Addis Ababa University. A quantitative cross-sectional study was conducted using structured questionnaires administered to postgraduate students and teaching faculty members in 13 MSc programs in biomedical and pharmaceutical sciences at the College of Health Sciences of Addis Ababa University. About 300 postgraduate and teaching faculty members were recruited with purposive sampling. The data collected included assessment methods, types of test items, and student preferences on assessment formats. Data were analyzed using quantitative approaches, descriptive statistics, and parametric tests. The study indicated that several assessment strategies and test items were practiced without a significant difference across fields of study. Regular attendance, oral questioning, quiz, group and individual assignments, seminar presentations, mid-term tests, and final written examination were commonly practiced assessment formats, while short question and long question essays were the most commonly used test items. However, students were not commonly assessed for skills and attitude. The students indicated they mostly preferred short essay questions, followed by practical-based examinations, long essay questions, and oral examination. The study identified several challenges to continuous assessment. Practice of assessing students' learning outcomes involves multiple methods focusing on assessing mainly knowledge; however, the assessment of skills appears inadequate, and several challenges appear to be hindering implementation of continuous assessment.

Adaptation of Assessment Techniques and Procedures of Music Learning Outcomes in The New Normal Era

The change in the Covid-19 pandemic era to the new normal era demands changes in the implementation of learning again. In this regard, this study aims to describe and analyze the adaptation of techniques and procedures for assessing learning outcomes carried out by teachers. The research was conducted with a qualitative descriptive method. Data was obtained through observation, interviews, and document study. The process of checking the validity of the data was done by triangulating data sources. The data analysis technique used descriptive analysis with the stages of data reduction, data presentation, and data verification. The study results show that the implementation of assessments in the Covid-19 pandemic era to the new normal era has changed. During the Covid-19 pandemic and the new normal, the knowledge assessment technique used was the same: the written test. However, the assessment procedures and types of test items were different. During the Covid 19 pandemic, the knowledge questions used were in the form of multiple choice, which was done digitally. Meanwhile, during the new normal, multiple-choice questions and essays were used with digital and manual (written) processing procedures. Skills assessment during the Covid-19 pandemic was carried out using a performance assessment whose object of observation was in the form of a video recording which was sent to a digital platform. Meanwhile, during the new normal, project evaluation was carried out to know the process and the results of the work which the procedures were carried out directly.

Gender Bias in Test Item Formats: Evidence from PISA 2009, 2012, and 2015 Math and Reading Tests

AbstractLarge‐scale standardized tests are regularly used to measure student achievement overall and for student subgroups. These uses assume tests provide comparable measures of outcomes across student subgroups, but prior research suggests score comparisons across gender groups may be complicated by the type of test items used. This paper presents evidence that among nationally representative samples of 15‐year‐olds in the United States participating in the 2009, 2012, and 2015 PISA math and reading tests, there are consistent item format by gender differences. On average, male students answer multiple‐choice items correctly relatively more often and female students answer constructed‐response items correctly relatively more often. These patterns were consistent across 34 additional participating PISA jurisdictions, although the size of the format differences varied and were larger on average in reading than math. The average magnitude of the format differences is not large enough to be flagged in routine differential item functioning analyses intended to detect test bias but is large enough to raise questions about the validity of inferences based on comparisons of scores across gender groups. Researchers and other test users should account for test item format, particularly when comparing scores across gender groups.

Exploring the Quality of Multiple-Choice Question Type of Test Items in Information and Communication Technology Using Item Analysis

The study explored the quality of multiple-choice question types of test items in Information and Communication Technology (ICT) using item analysis. The study was an exploratory type where secondary data was collected and analysed. The secondary data was made up of ten (10) Multiple Choice Questions (MCQ) items on an ICT test instrument. Thirty learners’ test instruments were sampled. The item analysis procedure included the item reliability coefficient, coefficient of variation, difficulty level, discrimination index, and distractor analysis. The study revealed that the test items had a low-reliability power and the class is homogenous. It was also found that ten per cent (10⁒) of the items were highly appropriate, fifty per cent (50⁒) needed revision, and forty per cent (40⁒) needed to be discarded. The study also found that some of the distractors were not appropriate and hence needed review while others need to be replaced. It was, therefore, recommended that item analysis should embrace race by teachers since it will foster effectiveness and efficiency carry out their duties in teaching and learning. Moreover, the Ministry of Education in Ghana must liaise with various heads of educational institutions to contract measurement and evaluation experts to organize workshops for their staff on how to do item analysis. Keywords: Reliability coefficient, Discrimination index, Difficulty level, Distractor analysis, Test items

Using Rasch analysis to examine raters’ expertise Turkish teacher candidates’ competency levels in writing different types of test items

The aim of the present study was to examine Turkish teacher candidates’ competency levels in writing different types of test items by utilizing Rasch analysis. In addition, the effect of the expertise of the raters scoring the items written by the teacher candidates was examined within the scope of the study. 84 Turkish teacher candidates participated in the present study, which was conducted using the relational survey model, one of the quantitative research methods. Three experts participated in the rating process: an expert in Turkish education, an expert in measurement and evaluation, and an expert in both Turkish education and measurement and evaluation. The teacher candidates wrote true-false, short response, multiple choice and open-ended types of items in accordance with the Test Item Development Form, and the raters scored each item type by designating a score between 1 and 5 based on the item evaluation scoring rubric prepared for each item type. The study revealed that Turkish teacher candidates had the highest level of competency in writing true-false items, while they had the lowest competency in writing multiple-choice items. Moreover, it was revealed that raters’ expertise had an effect on teacher candidates’ competencies in writing different types of items. Finally, it was found that the rater who was an expert in both Turkish education and measurement and evaluation had the highest level of scoring reliability, while the rater who solely had expertise in measurement and evaluation had the relatively lowest level of scoring reliability.

중학생 과학시험의 문항 유형에 따른 더닝-크루거 효과

Objectives In this study, the purpose of this study is to examine whether the Dunning-Kruger effect, which is a biased confidence in one's ability according to the type of question in the science subject, appears in middle school students, suggesting a learning method, and removing the negative factors of learning. The purpose of this study is to respond to the demands of the school to present a learning strategy that minimizes the discrepancy between academic expectations and academic achievement, and to prepare a basis for implementing it. Methods For 215 second-year coeducational middle school students located in Gyeongsangbuk-do, we had them predict the expected scores for each of the multiple-choice and short-answer types of test questions before the first science test in the second semester of 2021. A significant relationship was verified by analyzing the relationship between quartiles based on actual grades and expected grades, and t-testing. Results A significant negative correlation was confirmed between the expected score and the difference score (the gap between the expected score and the actual score), and it was also confirmed that the overestimation of the short-answer type was larger than that of the multiple-choice type question. By confirming the two-way phenomenon of overestimation of students with low academic achievement and underestimation of students with good grades, the Dunning-Kruger effect, a previous study, was also found in Korean middle school students. Conclusions The correlation between the student's actual achievement score and the difference score was negative (-) in both multiple-choice and short-answer types for each test item type. There is a significant difference between the actual grades and quartiles in estimating the expected scores for multiple-choice and short-answer types. We present a learning strategy to provide opportunities for self-directed learning that accurately judges one's own abilities, expand the question-and-answer-type question type method in the school field, and check whether there are factors that affect cognitive bias in addition to the question type.

Towards Integrated Testing Approach --- An Application of cognitive science and deep learning principle to language testing

The use of multiple-choice (MC) question types has been one of the most contentious issues in language testing. Much has been said and written about the use of MC over the years. However, no attempt has ever been made to introduce any innovation in test item types. The researchers proposed a jumbled words test item (JW) based on cognitive science and deep learning principles, and addressed the feasibility of replacing the type of multiple-choice (MC) question with JW to meet the ongoing rapid development of language testing practice. Two research questions were proposed ad hoc, focusing on the co-relationship between JW and MC scores. RASCH-GZ was used to perform item analyses (Rasch, 1960). The item difficulty parameters thus obtained were used to compare the two different test items. The sample data metric includes 40 Chinese participants. The findings revealed that correlation analysis revealed that the performance of the same group of subjects taking both JW and MC was not relevant (Pearson Corr = 0). This is primarily due to the total elimination of guessing factors inherent in test-takers during JW test performance. Three factors were specified for the design of the JW test: compute program, test difficulty, and score acceptability. These all have three dimensions. Data collected through questionnaires were analyzed using EFA in SPSS V.24.0. KMOs (=0.867) were found to be approximately one and significance at 0.000 (0.05), indicating that the construct of the Questionnaire thus designed has better validity for factor analysis. Three important conclusions were obtained, the implications of which could provide impetus for our testing counterparts to practice more precisely and correctly, potentially reshaping our overall language testing practice. Limitations and recommendations for future research were also discussed.

‘Time to figure out what to do’: understanding the nature of Irish post-primary students’ interactions with computer-based exams (CBEs) that use multimedia stimuli

ABSTRACT In line with the widespread proliferation of digital technology in everyday life, many countries are now beginning to use computer-based exams (CBEs) in their post-primary education systems. To ensure that these CBEs are delivered in a manner that preserves their fairness, validity, utility and credibility, several factors pertaining to their design and development will need to be considered. This research study investigated the extent to which the design of different types of test items (e.g. inclusion of multimedia stimuli) in a CBE can affect test-takers’ engagement and behaviour. Qualitative data from a cued-Retrospective Think Aloud (c-RTA) protocol were gathered from 12 participants who had participated in a previous eye-tracking study. Participants watched a replay of their eye movements and were asked to state out loud what they were thinking at different points of the replay. Thematic analysis of the responses from these cognitive interviews captured the nature of students’ interactions with online testing environments under three main themes: Familiarisation, Sense-making and Making Decisions. Students also provided their opinions of and recommendations for the future of Irish online assessments. These findings can offer guidelines to all stakeholders considering the use of CBEs in post-primary contexts.

DESIGN AND DEVELOPMENT OF A SOFTWARE MODULE FOR TESTING BASED ON THE MEDIAWIKI SERVER

The educational resources based on MediaWiki that provide the basic tools needed in the learning process was analyzed. After researching the MediaWiki extensions in use, we found two Extension Mediawiki Quizzer and Extension IntraACL extensions that allowed us to deploy a MediaWiki-based testing system. The approbation of the constructed system showed advantages and disadvantages. It is simpler and easier to create test items using wiki text markup. In the test, questions and answers may contain formulas, pictures, diagrams. The test is generated randomly from the available tasks. The answer options are displayed in random order. The test results are saved after the completion of the test in the form of a table. Test results can be printed or exported. The disadvantage of the system was the possibility to create only one type of test task with the choice of one correct answer. The developers were able to correct this weakness by creating their own software module for testing – CUSPUWikiTests (EasyTests) based on MediawikiQuizzer. The new extension for the MediaWiki server CUSPUWikiTests fully retains the benefits of the built-in MediaWiki system and allows you to create different types of test items: open answer test, single choice test, multiple choice test, ordering test, matching test. The implementation of the development was carried in the Central Ukrainian State Pedagogical University named after Volodymyr Vynnуchenko. A new site, testing.cuspu.edu.ua, was created, supplementing wiki.cuspu.edu.ua with knowledge control tools. At first, MediaWikiQuizzer was chosen as a module for testing, but its functionality is to be small, a new extension for testing was created, with more functionality CUSPUWikiTests.

The Preliminary Study for Development of Phoneme Perception Test Tool in the School-Aged Children

Purpose: The purpose of this study is to develop a test list for school-aged children that can evaluate consonant perception ability and verify the validity and difficulty.Methods: The test list consisted of 127 initial and 94 final consonant test items with a meaningful one-syllable words and was developed in two types for each difficulty level. To verify the validity of the developed phoneme perception test tool, twenty children with normal hearing were tested for the validity of the results and the contents were verified by experts in various fields.Results: As a result of comparing the rate of correct response according to the type of test item (initial vs. final consonant) and difficulty (easy vs. difficult item), the initial and easy responses were higher than the final and hard item, respectively, so the difficulty level of the test list was appropriate. In addition, there was no statistical difference in the rate of correct answers according to the test speakers, and the content validity analysis for experts verified that the developed test list was appropriate as a phoneme perception test tool for school-age children.Conclusion: The initial and final consonant test items developed through this study consisted of one target word and three examples which are a pair of minimum confrontation with the target word and are heteromorphic test lists consisting of easy and hard types according to the difficulty of the example. It could also be useful for not only the evaluation of the consonant perception ability but also rehabilitation.

Multi-station test scheduling optimization method for industrial robot servo system

This study examined the test scheduling optimization problem of industrial robot servo system (IRSS) samples with unequal numbers of different types of test items on the multiple IRSS comprehensive test platforms with the same function. In order to improve IRSS performance test efficiency, a multi-station test scheduling optimization method, which combines IRSS sample-level scheduling and test item-level scheduling, was proposed. First, the IRSS sample-level scheduling was carried out, and a model was established with reference to the identical parallel machine problem (IPMP). The optimal result of IRSS sample-level multi-station scheduling was obtained by solving the model. Second, based on the result of IRSS sample-level multi-station scheduling, and taking the ideal optimal test completion time of IRSS multi-station scheduling as the target, the test items at the stations were reallocated to obtain the optimal scheduling result. Finally, an application example was used to verify the efficiency of the proposed method, and the experiment result showed that the proposed method can effectively fulfill the optimal allocation of different IRSS test items of multiple IRSS samples on multiple IRSS comprehensive test platforms. The test time was 247 min shorter than the conventional sequential parallel test and only one min longer than the ideal optimal test completion time.

An Empirical Investigation of the Fairness of Multiple-Choice Items Relative to Constructed-Response Items on Tests of Students’ Mastery of Course Content

Although many instructors prefer multiple-choice (MC) items due to their convenience and objectivity, many others eschew their use due to concerns that they are less fair than constructed response (CR) items at evaluating student mastery of course content. To address three common unfairness concerns, I analyzed performance on MC and CR items from tests within nine sections of five different biology courses I taught over a five-year period. In all nine sections, students’ scores on MC items were highly correlated with their scores on CR items (overall r = 0.90), suggesting that MC and CR items quantified mastery of content in an essentially equivalent manner—at least to the extent that students’ relative rankings depended very little on the type of test item. In addition, there was no evidence that any students were unfairly disadvantaged on MC items (relative to their performance on CR items) due to poor guessing abilities. Finally, there was no evidence that females were unfairly assessed by MC items, as they scored 4% higher on average than males on both MC and CR items. Overall, there was no evidence that MC items were any less fair than CR items testing within the same content domain.

Тестирование учебных достижений: от проверки знаний к проверке понимания

The article presents the results of author’s research in the field of assessing the formation of competencies of students in educational institutions of higher education using test technologies. The analysis carried out in the context of pedagogical qualimetry allows us to draw conclusions about the high didactic potential of individual test tasks with the ordering of answers. The study of the features of the application of test technologies in the formation of funds for assessment tools for ongoing educational programs of higher education revealed the advantage of this tool, expressed in the form of the ability to differentiate grades de-pending on the didactic goals of the test and the depth of control of educational achievements. In the article, it is scientifically grounded that the use of single test tasks with ordering of answers, unlike other types of tests (for example, with a single choice, multiple choice, establishing compliance, establishing a sequence), provides the developer with significantly more possibilities in forming grading scale and at the same time provides it with more interesting for analysis results of monitoring student performance. The authors of the article obtained initial analytical data, with the help of which quantitative characteristics can be calculated, which, together with the relevant criteria, can evaluate the success of the test solution. The method of questioning students who were trained according to the advanced training program of the faculty “Innovative Educational Technologies” received estimates of various types of test items, it was confirmed that the development of tests based on single test items with ordering of answers requires not only deep knowledge in his subject, but also creative abilities and general outlook. Recommendations on the introduction of test technologies in the educational environment are given.

Developing Computer-Based Professional Competency Test Items For Automotive Engineering Teachers Professional Education

The objective of the study is to develop Computer-Based Professional Competency Test Items. This type of test items is very important for the successful performance of Teachers Professional Education, recalling the fact that the quality of Teachers Professional Education is determined by the Computer-Based National Test which dominant materials are related to Professional Competency. Then, the study was conducted by referring to the research and development model by Borg & Gall with simplification on several stages. After the study has been conducted, there are several findings that might be elaborated. First, the development of Computer-Based Professional Competency Test Items for the participants of Automotive Engineering Teachers Professional Education is conducted through several stages as follows: (a) planning (content, form and number of test items); (b) test items drafting; (c) computer program developing; (d) testing; and (e) evaluating. Second, based on the rational analysis the quality of the research and development results might be categorized as follows: (a) from the aspects of material there are 2 test items that have been incompatible for the Multiple Choice mode; (b) from the aspects of construction there are 5 test items that have been different formulation for the answers; and (c) from the aspects of language there is 1 test item that has incompatible standard for the appropriate Bahasa Indonesia. Third, the value of Difficulty Level might be described as follows: (a) 23 test items belong to “Easy” category; (b) 32 test items belong to “Moderate” category; and (c) 5 test items belong to “Difficult” category. It means that the ratio among the “Easy,” “Moderate,” and “Difficult” will be 38.301 : 53.301 : 8.401. Fourth, the value of Discriminant Capacity might be described as follows: (a) 2 test items belong to “Very Good” category; (b) 17 test items belong to “Good” category; (c) 13 test items belong to “Moderate” category; (d) 21 test items belong to “Poor” category; and (e) 7 test items belong to “Very Poor” category. It means that 28 test items (46.701) have not meet the requirement of a good test item. Fifth, the value of Distractor Effectiveness Distribution might be described as follows: (a) 3 test items have 1 ineffective distractor; (b) 15 items have 2 ineffective distractors; (c) 28 test items have 3 ineffective distractors; and (d) 14 test items have 4 ineffective distractors.

The Question of Choice in an Achievement

Achievement tests administered at national level play a significant role in the lives of test-takers as well as the whole society. This paper aims to investigate the effect of giving students a choice between two optional questions on their overall performance in a high-stake achievement test for university admission. It is hypothesized that questions targeting writing-based productive skills and language system necessitate display of abilities which are different from fact-based questions designed around story content. The two items are assumed to reflect different constructs that require different criteria of assessment. Consequently, the student’s overall score is affected by the item they choose to answer, which might not be reflective of their real language abilities. An open-ended interview was carried out with ten teachers working with grade 3 students in model secondary schools to investigate the nature of the two types of test items and their impact on the student’s performance. The data has proved that giving choice in an achievement test generates different performances that are assessed differently. It is recommended that in order to address the question of fairness, it is important to clearly define and balance the construct of the items that affect the student’s choice and performance.

THE EVALUATION OF NATIONAL EXAMINATION OF ENGLISH SUBJECT AT SECONDARY SCHOOL

The implementation of National Examination in Indonesia always becomes an interesting discussion both in education field and social life. One of the debates in the implementation of National Examination is about the fairness, many experts stated that the implementation of National Examination is unfair because the threeday exam is used to determine the success or failure of a student’s three years of high school. The second issue is about the quality of education in Indonesia’s school is gravely uneven. Some schools have luxury facilities and infrastructure, while others barely have a school building. So, to apply and expect the same standard from all of them is harsh and unrealistic. This study would like to analyze the questions in National Examination. The writer takes English test for science students as the primary data to answer the questions. The writer focuses on the principle of language assessment implemented in construction of English test in National Examination and the appropriateness of English test questions in National Examination based on the standardize of language assessment. The result shows that National examination in this case English test has been designed accordance to the principle of assessment. For standardized test of national examination, there are some types of test items are hardly to be found in the test. In terms of reading test items, multiple choices which are indicating to find reference words, phrase or sentence are not given in English test. It is dominated by finding out the detail or specific information about the text, implicit and general description about the text. Overall the English test in national examination has been design according to the standardized test. Only for several things need to be improved in term of test items variety in reading skill.

Student Achievement Based on the Use of Scientific Method in the Natural Science Subject in Elementary School

This research aims at investigating elementary school student achievement based on the use ofscientific method in teaching science from the test item types, as reflected by the item difficulty indexusing the classical test theory (CTT) and modern test theory (IRT). The first stage in developing the test was preparing the learning continuum of scientific method aspects by referring to the learning continuum of science process skill as developed by the previous existing research. In this research, the learning continuum was validated by expert judgment. As the tests were administered/carried out at the same time, four sets of tests were developed and administered to students of Grade 1 to 6 in Yogyakarta and Sleman Regency in the 2016-2017 school year. Samples were taken from three Technical Management Units (TMUs). Three TMUs were determined by observing the distribution of school locations from the center to the suburbs. The items were analyzed using CTT and IRT.The results of the research show that the student achievement reflected by item difficulty index based on CTT and IRT indicates the same level of category except for several sub-aspects. Those items from certain testsindicate higher difficulty level for Grade 4 to 6 students than for Grade 1 to 3 students. This case is not relevant to the expected learning outcomes.

개념도 활용 독해 전략이 영어 학습자들의 사실적 및 추론적 독해에 미치는 영향

개념도 활용 독해 전략이 영어 학습자들의 사실적 및 추론적 독해에 미치는 영향

Interaction of Previewing Reading Strategy with Item Types, Item Difficulty, Learning Styles, and Proficiency Levels

Despite the widespread use of the previewing strategy (i.e., previewing multiple choice options prior to reading a main passage) in a test context like CSAT English Reading, there are few studies that empirically identify its effects. To fill this gap, the current study examined the effect of previewing in relation with four interaction variables regarding test-items (item types; item difficulty) and learner’s characteristics (learning styles; English proficiency). The research was implemented based on the experimental design. Specifically, Korean secondary school students were assigned to either previewing or non-previewing condition. Also, four moderating variables were manipulated or measured through test-item choices and the survey. The collected data was analyzed through four series of Analysis of Covariance (ANCOVA). Main findings are as follows: First, the previewing strategy had significant influence on the reading performance. Especially, this study identified specific conditions for previewing to backfire. Second, the result clarified what type of test-item (e.g., inference type items) is appropriate for previewing. Third, it also told who should avoid using the previewing strategy (e.g., holistic style; intermediate level). The findings provided pedagogical implications on how to teach a specific reading strategy in the frame of Strategy-Based Instruction (SBI). Limitations and future research areas are suggested in the final chapter.

THE EVALUATION OF NATIONAL EXAMINATION OF ENGLISH SUBJECT AT SECONDARY SCHOOL

The implementation of National Examination in Indonesia always becomes an interesting discussion both in education field and social life. One of the debates in the implementation of National Examination is about the fairness, many experts stated that the implementation of National Examination is unfair because the three-day exam is used to determine the success or failure of a student’s three years of high school. The second issue is about the quality of education in Indonesia’s school is gravely uneven. Some schools have luxury facilities and infrastructure, while others barely have a school building. So, to apply and expect the same standard from all of them is harsh and unrealistic. This study would like to analyze the questions in National Examination. The writer takes English test for science students as the primary data to answer the questions. The writer focuses on the principle of language assessment implemented in construction of English test in National Examination and the appropriateness of English test questions in National Examination based on the standardize of language assessment. The result shows that National examination in this case English test has been designed accordance to the principle of assessment. For standardized test of national examination, there are some types of test items are hardly to be found in the test. In terms of reading test items, multiple choices which are indicating to find reference words, phrase or sentence are not given in English test. It is dominated by finding out the detail or specific information about the text, implicit and general description about the text. Overall the English test in national examination has been design according to the standardized test. Only for several things need to be improved in term of test items variety in reading skill