Difficulty Of Questions Research Articles

Docemur Docemus: Peer-Assisted Learning Improves the Knowledge Gain of Tutors in the Highest Quartile of Achievement but Not Those in the Lowest Quartile

Peer-assisted learning (PAL) is a form of collaborative learning where members of a peer group act as teachers for each other. A reciprocal PAL program was designed to investigate whether there were differential gains in knowledge acquisition among tutors compared with tutees. Bayesian statistical analysis was used to quantitatively assess the effect of tutor status on performance in a knowledge-based examination. Subgroup analysis according to student achievement and question difficulty was performed. Final year undergraduate medical students in a 5-year degree program (n = 126). The overall probability of getting a correct answer on the knowledge examination was 49.7%. For questions on topics where a student had acted as a tutor this improved to 57.3%. However, students who performed in the upper quartile had a greater percentage gain in the probability of a correct answer in topics that they had taught vs students who performed in the lowest quartile. There was demonstrable overall knowledge gain associated with acting as a tutor in a PAL program but the greatest gain occurred in students of highest academic ability.

A Comparison of Team-Based Learning Formats: Can We Minimize Stress While Maximizing Results?

Team-Based Learning (TBL) is a collaborative teaching method in which students utilize course content to solvechallenging problems. A modified version of TBL is used at the University of Louisville School of Medicine.Students complete questions on the Individual Readiness Assurance Test (iRAT) then gather in pre-assigned groupsto retake the quiz, given time to utilize their learning resources and discuss each of the questions (Team ReadinessAssurance Test-tRAT). Following this discussion, students take an Individual Summative Assessment Test (iSAT)with new questions at a similar cognitive level and content focus. While educational gains of TBL have been shown,student evaluations negatively assessed the teaching method with complaints regarding question difficulty and stresslevels. Thus, during implementation of TBL in the School of Dentistry, three main changes were made: (1) Thecontribution of TBL to the overall grade was reduced (2) TBL questions were cognitively aligned with unit examquestions, and (3) Scratch-off, lottery response cards were used to create a fun, game-like environment. This revisedTBL format, compared to the original format, resulted in similar student performance during iRAT and tRATsessions. However, the revised, low-stress format had significantly higher scores on the iSAT (n=119-161, p <.05).Furthermore, students participating in the revised TBL format reported higher effectiveness of the learning format,higher levels of perceived fairness, and lower stress levels. These results suggest that the qualitative experience ofstudents may be an important consideration that should be carefully evaluated during implementation of a newteaching technique.

The evaluation of educational measurement indexes from the Undergraduates Clinical Skills Competition in a medical university

Objective The purpose of this study was to analyze and evaluate the educational measurem ent indexes from the undergraduates clinical skills competition in a medical university. The rationality of the competition was discussed in the view of education survey. Methods 84 students' results which selected from the China Medical University 2nd Clinical Skills Competition were the research objects in this study. The indexes of the difficulty, discrimination, reliability and criterion-related validity of the competition were evaluated based on the Education surveying principle. The difficulty was calculated by subjective topic difficulty computing formula, and other indexes were carried out with SPSS 18.0 except the difficulty index. Results The average difficulty of the questions of this university was 0.61, the average discrimination was 0.77 and the overall reliability was 0.926. The OSCE scores of the participating students were taken as the criterion in this study. The calculated criterion-related validity was 0.416. Conclusion The college students' clinical skills competition has good reliability, discrimination and effective criterion-related validity. For the selective competition, the difficulty can be further improved while considering the scope of the topic. Key words: Clinical skills competition; Reliability; Difficulty; Discrimination; Criterionrelated validity

A Comparison of Team‐Based Learning Formats: Can We Minimize Stress While Maximizing Results

Team‐Based Learning (TBL) is used at the University of Louisville School of Medicine as a collaborative teaching method in which students solve challenging physiology problems. Students take an Individual Readiness Assessment Test (iRAT), Team Readiness Assessment Test (tRAT), and Individual Summative Assessment Test (iSAT). While educational gains of TBL have been shown, some student evaluations negatively assessed the teaching method with complaints regarding question difficulty and stress levels. Thus, during implementation of TBL in the School of Dentistry, two changes were made: (1) TBL questions were cognitively aligned with unit exam questions (2) Scratch‐off, lottery response cards were used to create a fun, game‐like environment. This revised TBL format, compared to the original TBL format, resulted in similar student performance during iRAT and tRAT sessions. However, the revised, low‐stress format had significantly higher scores on the iSAT (n=119‐161, p&lt;.05). Furthermore, on Likert‐scale surveys, students participating in the revised TBL format subjectively reported higher effectiveness of the learning format, higher levels of perceived fairness, and lower stress levels. These results suggest that the qualitative experience of students, when using a new teaching format such as TBL, may be an important consideration that should be carefully evaluated during implementation.

Volumetric CT-images improve testing of radiological image interpretation skills

Rationale and objectivesCurrent radiology practice increasingly involves interpretation of volumetric data sets. In contrast, most radiology tests still contain only 2D images. We introduced a new testing tool that allows for stack viewing of volumetric images in our undergraduate radiology program. We hypothesized that tests with volumetric CT-images enhance test quality, in comparison with traditional completely 2D image-based tests, because they might better reflect required skills for clinical practice. Materials and methodsTwo groups of medical students (n=139; n=143), trained with 2D and volumetric CT-images, took a digital radiology test in two versions (A and B), each containing both 2D and volumetric CT-image questions. In a questionnaire, they were asked to comment on the representativeness for clinical practice, difficulty and user-friendliness of the test questions and testing program. Students’ test scores and reliabilities, measured with Cronbach's alpha, of 2D and volumetric CT-image tests were compared. ResultsEstimated reliabilities (Cronbach's alphas) were higher for volumetric CT-image scores (version A: .51 and version B: .54), than for 2D CT-image scores (version A: .24 and version B: .37). Participants found volumetric CT-image tests more representative of clinical practice, and considered them to be less difficult than volumetric CT-image questions. However, in one version (A), volumetric CT-image scores (M 80.9, SD 14.8) were significantly lower than 2D CT-image scores (M 88.4, SD 10.4) (p<.001). The volumetric CT-image testing program was considered user-friendly. ConclusionThis study shows that volumetric image questions can be successfully integrated in students’ radiology testing. Results suggests that the inclusion of volumetric CT-images might improve the quality of radiology tests by positively impacting perceived representativeness for clinical practice and increasing reliability of the test.

Exploring the difficulty of mathematics examination questions for weaker readers

This research investigated the difficulty of examination questions for students with weaker reading skills. Item level performance data were obtained for all candidates who took a maths examination (for 16 year olds). A sub-group of students who had access to a reader was identified (students with proven reading difficulties are permitted to have an adult available to read the examination questions aloud). A matched control group of students was also identified. Facility values were calculated for the “Reader” and “Matched” groups and used to compare item difficulty between groups. Questions for which there was a statistically significant difference between scores for the two groups were selected for more detailed analysis of the responses given by 100 candidates from each group. The analyses attempted to identify the question features likely to have contributed to relative difficulty or ease for those with weaker reading skills. This can inform future question-writing practice.

Signaling smarts? Revealed preferences for self and social perceptions of intelligence

We design a laboratory experiment to test for image motives in a setting where decisions signal intelligence to a social audience. Money-maximizing behavior in the experiment sorts subjects by academic ability, as measured by performance on verbal analogy questions, across two levels of question difficulty. Sorting behavior is publicly revealed in our “audience” treatment, facilitating social signaling. In the “intrinsic only” treatment, the signaling mechanism was explained but decisions were kept private. In the control, there was no discussion of the signaling mechanism and all decisions were private. We find that intrinsic only subjects were significantly more likely to choose the high-type action than the control. In comparison, subjects were significantly less likely to choose the signal in the audience treatment, when doing so was publicly observed. The effects are more pronounced in males. The results suggest that social observation can demotivate individuals when decisions signal intelligence, despite evidence that the underlying trait was privately considered desirable. Audience effects have a less predictable impact on behavior in this setting as compared to the near universally positive findings from the altruism and trust literature. Our experimental design can be easily adapted to study image motives in a broad set of environments using revealed preferences.

Signaling Smarts? Revealed Preferences for Self and Social Perceptions of Intelligence

We design a laboratory experiment to test for image motives in a setting where decisions signal intelligence to a social audience. Money-maximizing behavior in the experiment sorts subjects by academic ability, as measured by performance on verbal analogy questions, across two levels of question difficulty. Sorting behavior is publicly revealed in our “audience” treatment, facilitating social signaling. In the “intrinsic only” treatment, the signaling mechanism was explained but decisions were kept private. In the control, there was no discussion of the signaling mechanism and all decisions were private. We find that intrinsic only subjects were significantly more likely to choose the high-type action than the control. In comparison, subjects were significantly less likely to choose the signal in the audience treatment, when doing so was publicly observed. The effects are more pronounced in males. The results suggest that social observation can demotivate individuals when decisions signal intelligence, despite evidence that the underlying trait was privately considered desirable. Audience effects have a less predictable impact on behavior in this setting as compared to the near universally positive findings from the altruism and trust literature. Our experimental design can be easily adapted to study image motives in a broad set of environments using revealed preferences.

Teachers of Students with Visual Impairments and Their Use of Assistive Technology: Measuring the Proficiency of Teachers and Their Identification with a Community of Practice

Introduction This article presents an instrument that measures the assistive technology proficiency of teachers of students with visual impairments and their identification with a community of practice that values assistive technology. Teachers’ deficits in assistive technology proficiency negatively impact students who are visually impaired by stunting the development of assistive technology skills, ultimately resulting in poorer postsecondary education and employment outcomes. Identification with a community of practice that values assistive technology may be supportive of the technological proficiency of teachers of students with visual impairments. Method Assistive technology proficiency and community of practice identification dimensions were defined and outlined in rubric-like “construct maps.” A survey that was created to place teachers of students with visual impairments within each construct map was completed by 33 Californian teachers. Survey performance was evaluated by estimating Rasch models, which provided information on relative question difficulty and question performance. Results Estimated question difficulties revealed expected patterns. Only two survey questions performed irregularly (infit &gt; 1.33). Internal reliability was good (Cronbach's Alpha = 0.80) for assistive technology proficiency, and acceptable (Cronbach's Alpha = 0.70) for community of practice identification. Discussion The findings suggest the survey reliably measured the assistive technology proficiency and identification with a community of practice that values technology in this sample of teachers. Utilization of this tool may enable the objective evaluation of assistive technology proficiency of teachers pre- and post-training. Implications for practitioners Creation of a reliable instrument that measures these constructs will support investigations in how one relates to the other, and will consider how professional development may be designed to better support the use of assistive technology by teachers.

Graphical Logic Paradigm and Reasoning Process Research

Through the mental models, mental logic and raven's graphic reasoning test research, this paper deduced a series of graphic reasoning paradigm, and summarized the process of graphical reasoning. Through the demonstration of problem difficulty experiment and written report experiment, confirmed a high relevance between the graphic logic steps iteration and species and questions difficulty. Furthermore, reasoning order and default graphical reasoning paradigm have a high degree of agreement. Confirmed the rationality and widespread adaptability of graphic reasoning paradigms.

A Comparison of Students, Scores in Multiple- Choice and Essay Questions in Mother and Child Health Examination

Background: The method of evaluation in students’ learning affects their studying mode. The aim of this study was to compare nursing students’ scores in essay and multiple-choice questions in Mother and Child Health examination in Shahrekord University of Medical Sciences. Methods: In this study, the final scores of 121 of nursing students in a multiple-choice questionnaire consisting of 40 questions, during four semesters were compared with the scores obtained from an essay questionnaire composed of 40 questions. Difficulty and Discrimination Index of questions were similar in both kinds of questions. Data was analyzed employing paired t-test and Pearson correlation test and PResults: Mean and standard deviation of students’ scores were reported 14.72±3.7 in essay questions and 16.53±2.4 in multiple-choice questions and showed a significant difference (P<0.001). A positive correlation was reported between the essay and multiple-choice questions (r=0.93). Students’ scores were higher in multiple-choice questionnaire compared to the essay one. Using multiple-choice questionnaire on its own may lead to the success of those students who do not have the ability of perceive, resolve and apply.

Metacognitive effects of initial question difficulty on subsequent memory performance

In two experiments, we examined whether relative retrieval fluency (the relative ease or difficulty of answering questions from memory) would be translated, via metacognitive monitoring and control processes, into an overt effect on the controlled behavior-that is, the decision whether to answer a question or abstain. Before answering a target set of multiple-choice general-knowledge questions (intermediate-difficulty questions in Exp. 1, deceptive questions in Exp. 2), the participants first answered either a set of difficult questions or a set of easy questions. For each question, they provided a forced-report answer, followed by a subjective assessment of the likelihood that their answer was correct (confidence) and by a free-report control decision-whether or not to report the answer for a potential monetary bonus (or penalty). The participants' ability to answer the target questions (forced-report proportion correct) was unaffected by the initial question difficulty. However, a predicted metacognitive contrast effect was observed: When the target questions were preceded by a set of difficult rather than easy questions, the participants were more confident in their answers to the target questions, and hence were more likely to report them, thus increasing the quantity of freely reported correct information. The option of free report was more beneficial after initial question difficulty than after initial question ease, in terms of both the gain in accuracy (Exp. 2) and a smaller cost in quantity (Exps. 1 and 2). These results demonstrate that changes in subjective experience can influence metacognitive monitoring and control, thereby affecting free-report memory performance independently of forced-report performance.

The relationship between non-functioning distractors and item difficulty of multiple choice questions: A descriptive analysis

Objectives: This study investigates the relationship between the presence of non-functioning distractors (NFDs) and the difficulty index of multiple choice questions (MCQs). Materials and Methods: The number of NFDs and difficulty index were correlated for each MCQ of Family Medicine Examination, College of Medicine, King Saud University (KSU) in three consecutive years (2010, 2011 and 2012). Results: The MCQs with more NFDs showed high difficulty index as compared to MCQs having less NFDs. The pattern of increasing difficulty index was 1NFD Conclusion: MCQs with a higher number of NFDs are easier than those with lower number of NFDs.

New Indices for Refining Multiple Choice Questions

Multiple choice questions (MCQs) are one of the most popular tools to evaluate learning and knowledge in higher education. Nowadays, there are a few indices to measure reliability and validity of these questions, for instance, to check the difficulty of a particular question (item) or the ability to discriminate from less to more knowledge. In this work two new indices have been constructed: (i) the no answer index measures the relationship between the number of errors and the number of no answers; (ii) the homogeneity index measures homogeneity of the wrong responses (distractors). The indices are based on the lack-of-fit statistic, whose distribution is approximated by a chi-square distribution for a large number of errors. An algorithm combining several traditional and new indices has been developed to refine continuously a database of MCQs. The final objective of this work is the classification of MCQs from a large database of items in order to produce an automated-supervised system of generating tests with specific characteristics, such as more or less difficulty or capacity of discriminating knowledge of the topic.

RELACIJE IZMEĐU ISPITNE ANKSIOZNOSTI I AKADEMSKOG POSTIGNUĆA

U velikoj većini današnjih društava skoro je nemoguće odrasti bez susreta sa mnogobrojnim proverama različitih personalnih atributa, poput sposobnosti, znanja, veština, pa čak i određenih karakteristika ličnosti. Gotovo da ne postoji oblast života koja ne podrazumeva barem neki oblik procene naše vrednosti, a mnoge od tih evaluacija imaju prilično ozbiljne posledice po našu budućnost. Uzimajući u obzir značajne konsekvence raznih ispitnih situacija, ne iznenađuje činjenica da su one u velikoj meri nabijene visokim stresogenim potencijalom. U okviru ovog preglednog članka prvo je dat prikaz savremenih konceptualizacija ispitne anksioznosti i opisana je struktura ovog izuzetno rasprostranjenog fenomena. Drugi deo rada posvećen je pregledu mnogobrojnih empirijskih činjenica o odnosima koji postoje između ispitne anksioznosti, s jedne strane, i uspeha u evaluativnim situacijama, s druge. Posebna pažnja posvećena je varijablama koje imaju ulogu medijatora i moderatora u ovom specifičnom i složenom stres-procesu.

Are Readability Formulas Valid Tools for Assessing Survey Question Difficulty?

Readability formulas, such as the Flesch Reading Ease formula, the Flesch–Kincaid Grade Level Index, the Gunning Fog Index, and the Dale–Chall formula are often considered to be objective measures of language complexity. Not surprisingly, survey researchers have frequently used readability scores as indicators of question difficulty and it has been repeatedly suggested that the formulas be applied during the questionnaire design phase, to identify problematic items and to assist survey designers in revising flawed questions. At the same time, the formulas have faced severe criticism among reading researchers, particularly because they are predominantly based on only two variables (word length/frequency and sentence length) that may not be appropriate predictors of language difficulty. The present study examines whether the four readability formulas named above correctly identify problematic survey questions. Readability scores were calculated for 71 question pairs, each of which included a problematic (e.g., syntactically complex, vague, etc.) and an improved version of the question. The question pairs came from two sources: (1) existing literature on questionnaire design and (2) the Q-BANK database. The analyses revealed that the readability formulas often favored the problematic over the improved version. On average, the success rate of the formulas in identifying the difficult questions was below 50 percent and agreement between the various formulas varied considerably. Reasons for this poor performance, as well as implications for the use of readability formulas during questionnaire design and testing, are discussed.

Improving Multiple–Choice Questions to Better Assess Dental Student Knowledge: Distractor Utilization in Oral and Maxillofacial Pathology Course Examinations

How many incorrect response options (known as distractors) to use in multiple-choice questions has been the source of considerable debate in the assessment literature, especially relative to influence on the likelihood of students' guessing the correct answer. This study compared distractor use by second-year dental students in three successive oral and maxillofacial pathology classes that had three different examination question formats and scoring resulting in different levels of academic performance. One class was given all multiple-choice questions; the two other were given half multiple-choice questions, with and without formula scoring, and half un-cued short-answer questions. Use by at least 1 percent of the students was found to better identify functioning distractors than higher cutoffs. The average number of functioning distractors differed among the three classes and did not always correspond to differences in class scores. Increased numbers of functioning distractors were associated with higher question discrimination and greater question difficulty. Fewer functioning distractors fostered more effective student guessing and overestimation of academic achievement. Appropriate identification of functioning distractors is essential for improving examination quality and better estimating actual student knowledge through retrospective use of formula scoring, where the amount subtracted for incorrect answers is based on the harmonic mean number of functioning distractors.

Two Methods for Measuring Question Difficulty and Discrimination in Incomplete Crowdsourced Data

Assistance in creating high-quality exams would be welcomed by educators who do not have direct access to the proprietary data and methods used by educational testing companies. The current approach for measuring question difficulty relies on models of how good pupils will perform and contrasts that with their lower-performing peers. Inverting this process and allowing educators to test their questions before students answer them will speed up question development and utility. We cover two methods for automatically judging the difficulty and discriminating power of MCQs and how best to build sufficient exams from good questions.

WHO REALLY WANTS TO BE A MILLIONAIRE? ESTIMATES OF RISK AVERSION FROM GAMESHOW DATA

There is a considerable variation in estimates of the degree of risk aversion in the literature. This paper analyses the behaviour of contestants in one of the most popular TV gameshows ever to estimate a CRRA model of behaviour. This gameshow has a number of features that makes it well suited for our analysis: the format is extremely straightforward, it involves no strategic decision-making, we have a large number of observations, and the prizes are cash and paid immediately, and cover a large range - up to £1 million. Our data sources have the virtue that we are able to check the representativeness of the gameshow participants. While the game requires skill, which complicates our analysis, the structure of the game is very simple so that complex probability calculations are not required of the participants. The CRRA model is complex despite its restrictiveness because of the sequential nature of this game - answering a question correctly opens the option to hear the next question and this has a value that depends on the stage of the game and the player's view about the difficulty of subsequent questions. We use the data to estimate the degree of risk aversion and how it varies across individuals. We investigate a number of departures from this simple model including allowing the RRA parameter to vary by gender and age. Even though the model is extremely restrictive, in particular, it features a single RRA parameter we find that it fits the data across a wide range of wealth remarkably well and yields very plausible parameter values.

Anomalous propagation of acoustic traveling waves in thermoviscous fluids under the Rubin–Rosenau–Gottlieb theory of dispersive media

We report on an apparent anomaly associated with the compressible, thermoviscous version of what has come to be known as RRG theory. The difficulty in question, which does not appear to impact the lossless special case of this theory, manifests itself under the traveling wave assumption and has the shock-front of the resulting kink waveform and the (instigating) piston propagating in opposite directions.