Extended Matching Questions Research Articles

English

Multiple choice questions (MCQs) are a popular method of assessment due to its sheer advantage of ability to evaluate a lot of candidate in a short span of time. AIMS & OBJECTIVE: However due to different varieties of MCQs available there seems to be lacunae in the scoring pattern of MCQ in comparison to other methods of evaluation. METHODS: 100 MBBS students were taken and marks obtained in MCQs, written and practical assessment were evaluated and percentage of each was calculated. RESULTS: The marks obtained in MCQs were significantly higher as compared to written and practical scores. CONCLUSION: MCQs are scoring as compared to other methods of assessment and pass marks in MCQs should be higher as compared to other methods of evaluation. Also Extended matching questions (EMQs) should be used in place of MCQs.

Multiple choice questions are superior to extended matching questions to identify medicine and biomedical sciences students who perform poorly.

In recent years, medical faculties at Dutch universities have implemented a legally binding study advice to students of medicine and biomedical sciences during their propaedeutic phase. Appropriate examination is essential to discriminate between poor (grade <6), moderate (grade 6–8) and excellent (grade ≥8) students. Therefore, we compared the discriminatory properties of extended matching questions (EMQs) versus multiple-choice questions (MCQs) and identified the role of sex, age and examination preference on this score. Data were collected for 452 first-year medical and biomedical science students during three distinct course examinations: one examination with EMQ only, one with MCQ only and one mixed examination (including EMQ and MCQ). Logistic regression analysis revealed that MCQ examination was 3 times better in identifying poor students compared with EMQ (RR 3.0, CI 2.0–4.5), whereas EMQ better detected excellent students (average grade ≥8) (RR 1.93, CI 1.47–2.53). Mixed examination had comparable characteristics to MCQ. Sex and examination preference did not impact the score of the student. Students ≥20 years had a 4-fold higher risk ratio of obtaining a poor grade (<6) compared with students ≤18 years old (RR 4.1, CI 2.1–8.0). Given the strong discriminative capacity of MCQ examinations to identify poor students, we recommend the use of this type of examination during the propaedeutic phase of medicine and biomedical science study programmes, in the light of the binding study advice.

Exploring the validity of assessment in anatomy: Do images influence cognitive processes used in answering extended matching questions?

Assessment is an important aspect of medical education because it tests students' competence and motivates them to study. Various assessment methods, with and without images, are used in the study of anatomy. In this study, we investigated the use of extended matching questions (EMQs). To gain insight into the influence of images on the validity of test items, we focused on students' cognitive processes while they answered questions with and without images. Seventeen first-year medical students answered EMQs about gross anatomy, combined with either labeled images or answer lists, while thinking aloud. The participants' verbal reports were transcribed verbatim and then coded. Initial codes were based on a task analysis and were adapted into final codes during the coding process. Results showed that students used more cues from EMQs with images and visualized more often in EMQs with answer lists. Ready knowledge and verbal reasoning were used equally often in both conditions. In conclusion, EMQs with and without images elicit different results in this think aloud experiment, indicating different cognitive processes. They seem to measure different skills, making them valid for different testing purposes. The take-home message for anatomy teachers is that questions without images seem to test the quality of students' mental images while questions with images test their ability to interpret visual information. It makes sense to use both response formats in tests. Using images from clinical practice instead of anatomical drawings will help to improve test validity.

Succeeding in paediatric surgery examinations, volumes 1 and 2

The editors indicate that these two volumes are a ‘complete resource … all that is essential for succeeding in paediatric surgical examinations’. Subsequent to examinations, they can be used as a reference book. The two books are laid out in an identical fashion, each with 84 chapters dividing paediatric and neonatal surgery into conventional sections, largely based on diagnosis. Most chapters in Volume 1 provide 15–25 multiple choice questions, some more. These are supported by comprehensive answers and explanations, and do not leave the reader feeling that the questions have been left only partially answered. In Volume 2, the same authors return to their subjects, this time in various guises of extended matching questions (5–10 per chapter). Some care has clearly been taken to avoid simply asking the same questions (that appeared in Vol 1) in the same way. As a result, taking both books together, considerable ground is covered in each of the 84 chapters. These volumes should be approached as a source of vignettes of information rather than question books. In these days of highly crafted national examinations, some of the questions and answers are, at best, ambiguous and some questions could be completed using more than one of the available answers. They would not survive the scrutiny that our current question bank demands. Nevertheless, the grandiose editorial claim may not be too far from the truth. If the questions and accompanying text are viewed as a series of challenges, stimulating discussion and background reading, then the knowledge essential for succeeding in examinations should be covered. Although sometimes irritatingly disagreeable, the questions and answers are reminiscent of the cut and thrust of the dogmatic (but not invariably correct) senior consultant surgeon’s ward round. The work carried out subsequently to either confirm or refute his or her assertions was of immense value in learning the subject and these books will achieve the same objective. This book is recommended to all paediatric surgeons, at any stage of their career.

Assessing the Clinical Skill of Neurological Localization Using Extended-Matching Questions (S27.001)

Assessing the Clinical Skill of Neurological Localization Using Extended-Matching Questions (S27.001)

Multimedia extended matching questions (EMQs) in undergraduate musculoskeletal education

Multimedia extended matching questions (EMQs) in undergraduate musculoskeletal education

How do students with dyslexia perform in extended matching questions, short answer questions and observed structured clinical examinations?

There are an increasing number of students with learning difficulties attending university, and currently much debate about the suitability and ability of students with dyslexia at both medical school and once they graduate into clinical practice. In this study we describe the performance of students with dyslexia compared to fellow students in extended matching questions (EMQ), short answer question (SAQ) and observed structured clinical examinations (OSCE) and discuss the implications of differences identified. End of year assessment results for 5 cohorts of medical students were analysed. Students with dyslexia did less well overall in all assessment types in year 1 but this difference was not evident in later years. Dyslexic students who were allowed extra time in written assessments did better than dyslexic students who did not have their assessment concessions in place. When station type within OSCE assessments was analysed students with dyslexia did less well in both examination skills and data interpretation stations in years 1, 2 & 3. In conclusion, differences in performance in written assessments are only evident early in training and may be partly due to delayed adjustment to medical school or implementation of assessment concessions. Performance in individual OSCE stations is dependent on station type. Why students with specific learning difficulties (SpLDs) perform less well in examination skills and data analysis OSCE stations requires further investigation.

The performance of junior doctors in applying clinical pharmacology knowledge and prescribing skills to standardized clinical cases

Recent studies suggest a worryingly high proportion of final year medical students and new doctors feel unprepared for effective and safe prescribing. Little research has been undertaken on UK junior doctors to see if these perceptions translate into unsafe prescribing practice. We aimed to measure the performance of foundation year 1 (FY1) doctors in applying clinical pharmacology and therapeutics (CPT) knowledge and prescribing skills using standardized clinical cases. A subject matter expert (SME) panel constructed a blueprint, and from these, twelve assessments focusing on areas posing high risk to patient safety and deemed as essential for FY1 doctors to know were chosen. Assessments comprised six extended matching questions (EMQs) and six written unobserved structured clinical examinations (WUSCEs) covering seven CPT domains. Two of each assessment types were administered over three time points to 128 FY1 doctors. The twelve assessments were valid and statistically reliable. Across seven CPT areas tested 51-75% of FY1 doctors failed EMQs and 27-70% failed WUSCEs. The WUSCEs showed three performance trends; 30% of FY1 doctors consistently performing poorly, 50% performing around the passing score, and 20% performing consistently well. Categorical rating of the WUSCEs revealed 5% (8/161) of scripts contained errors deemed as potentially lethal. This study showed that a large proportion of FY1 doctors failed to demonstrate the level of CPT knowledge and prescribing ability required at this stage of their careers. We identified areas of performance weakness that posed high risk to patient safety and suggested ways to improve the prescribing by FY1 doctors.

Is case-specificity content-specificity? An analysis of data from extended-matching questions

Case-specificity, i.e., variability of a subject's performance across cases, has been a consistent finding in medical education. It has important implications for assessment validity and reliability. Its root causes remain a matter of discussion. One hypothesis, content-specificity, links variability of performance to variable levels of relevant knowledge. Extended-matching items (EMIs) are an ideal format to test this hypothesis as items are grouped by topic. If differences pertaining to content knowledge are the main cause of case-specificity, variability across topics should be high and variability across items within the same topic low. We used generalisability analysis on results of a written test composed of 159 EMIs sat by two cohorts of general practice trainees at one university. Two hundred and twenty-seven trainees took part. The variance component attributed to subjects was small. Variance attributed to topics was smaller than variance attributed to items. The main source of error was interaction between subjects and items, accounting for two-thirds of error. The generalisability D study revealed that for the same total number of items, increasing the number of topics results in a higher G coefficient than increasing the number of items per topic. Topical knowledge does not seem to explain case-specificity observed in our data. Structure of knowledge and reasoning strategy may be more important, in particular pattern-recognition which EMIs were designed to elicit. The causal explanations of case-specificity may be dependent on test format. Increasing the number of topics with fewer items each would increase reliability but also testing time.

Erratum to: Is case-specificity content-specificity? An analysis of data from extended-matching questions

Erratum to: Is case-specificity content-specificity? An analysis of data from extended-matching questions

How to write and how to answer EMQs

The shift to extended matching questions means that some people setting or sitting college examinations will be faced with an unfamiliar assessment method. This article briefly outlines the process of examination construction, the advantages and disadvantages of extended matching questions and an approach to such questions when writing and answering examination papers.

Attitudes of Veterinary Faculty to the Assessment of Clinical Reasoning Using Extended Matching Questions

For assessment purposes, clinical expertise is often divided into three broad components: scientific and clinical knowledge, clinical reasoning, and practical/technical skills. This structure can be used to define the tools used for assessment of clinical students. Knowledge can be assessed through a variety of written formats and skills through various practical assessments, including the objective structured clinical examination (OSCE), but the assessment of clinical reasoning has proved to be far more challenging. A companion paper (Tomlin JL, Pead MJ, May SA. Veterinary students' attitudes toward the assessment of clinical reasoning using extended matching questions. J Vet Med Educ 35:612-621, 2008) reports on the identification and implementation of a valid and reliable method to assess clinical reasoning using clinical-scenario-based extended matching questions (EMQs) in the final examinations at the Royal Veterinary College and looks at students' response to the new examination format. Although EMQs were generally well accepted, many students were concerned about the implied encouragement of pattern recognition, a non-analytical form of clinical reasoning that results from recognition of familiar clinical situations. This paper addresses the attitudes of the teaching faculty to the EMQ format. The students' concerns about promotion of pattern recognition, was also explored in more depth. Overall, faculty perceived EMQs as an appropriate way to test clinical reasoning and as relevant to the experience that students would have gained during their clinical rotations. However, faculty felt that EMQs were difficult to write and that poorly written questions tended to promote pattern recognition. Almost half reiterated the students' concerns that pattern recognition may be an inappropriate reasoning strategy for undergraduates.

Measurement Characteristics of Content-Parallel Single-Best-Answer and Extended-Matching Questions in Relation to Number and Source of Options

Previous research showed that extended-matching questions (EMQs) with eight options per set resulted in better score precision than EMQs with larger numbers of options or independent single-best-answer items (A-type) with five options. This study extends previous work using smaller numbers of options. Ninety-six questions were presented in two formats on United States Medical Licensing Examination Step 2: as two-item EMQ sets and as independent A-types. Four versions of EMQs were used: five- and eight-option versions with options selected using statistics, and five- and eight-option versions with options selected by physicians. Seven A-type versions were used: three-, four-, five-, and eight-option versions with options selected using statistics, and three-, four-, and five-option versions with options selected by physicians. Items with more options were harder, required more time to complete, and had similar item discrimination. Option sets selected by physicians were easier, slightly more discriminating, and required less testing time. A-types with four or five options and EMQs with eight options make more efficient use of testing time. Provision of response statistics to content experts does not seem necessary to guide option selection.

Designing assessment of pathology in the undergraduate curriculum

The teaching and learning of pathology in undergraduate curricula are changing and the methods of assessment of the pathology component need to change with them. The curricular changes from the older, traditional programmes to newer, more integrated courses, often using a problem-based learning (PBL) approach, are a challenge for pathologists at a time of reduction of academic pathology posts. However, pathologists have a broad range of knowledge and skills that are invaluable in undergraduate teaching, learning and assessment. These should be encouraged in training grades and promoted by established pathologists as ‘role models’. Newer forms of assessment provide an opportunity for pathologists to maintain the profile of the discipline since ‘assessment drives learning’. These methods include the incorporation of well-designed clinical scenarios with pathological relevance into multiple choice questions (MCQs) and extended matching questions (EMQs), examples of which are given. Similarly, objective structured clinical examinations (OSCEs) also allow the use of clinically relevant settings such as death certification and multidisciplinary team meetings in undergraduate examinations. Pathologists should get involved in all levels of assessment, especially in question writing and quality assurance. They should also participate in OSCEs and function as external examiners to maintain the profile of pathology to undergraduates.

BOOK REVIEWS

Like it or not, extended matching questions (EMQs) are becoming increasingly popular as a method of assessment in medical school examinations. This book is an excellent revision aid for medical students sitting for Finals as it essentially covers the entire surgical syllabus. The format of the questions closely resemble to those found in the University Medical Assessment Partnership bank of questions and as such serve as a useful preparation for exams.

Introduction to the Extended Matching Questions (EMQs)

Introduction to the Extended Matching Questions (EMQs)

Answers to MCQ

The teaching and learning of pathology in undergraduate curricula are changing and the methods of assessment of the pathology component need to change with them. The curricular changes from the older, traditional programmes to newer, more integrated courses, often using a problem-based learning (PBL) approach, are a challenge for pathologists at a time of reduction of academic pathology posts. However, pathologists have a broad range of knowledge and skills that are invaluable in undergraduate teaching, learning and assessment. These should be encouraged in training grades and promoted by established pathologists as ‘role models’. Newer forms of assessment provide an opportunity for pathologists to maintain the profile of the discipline since ‘assessment drives learning’. These methods include the incorporation of well-designed clinical scenarios with pathological relevance into multiple choice questions (MCQs) and extended matching questions (EMQs), examples of which are given. Similarly, objective structured clinical examinations (OSCEs) also allow the use of clinically relevant settings such as death certification and multidisciplinary team meetings in undergraduate examinations. Pathologists should get involved in all levels of assessment, especially in question writing and quality assurance. They should also participate in OSCEs and function as external examiners to maintain the profile of pathology to undergraduates.

Computer-based testing of the modified essay question: the Singapore experience

Background: The modified essay question (MEQ), featuring an evolving case scenario, tests a candidate's problem-solving and reasoning ability, rather than mere factual recall. Although it is traditionally conducted as a pen-and-paper examination, our university has run the MEQ using computer-based testing (CBT) since 2003.Aims: We describe our experience with running the MEQ examination using the IVLE, or integrated virtual learning environment (https://ivle.nus.edu.sg), provide a blueprint for universities intending to conduct computer-based testing of the MEQ, and detail how our MEQ examination has evolved since its inception.Methods: An MEQ committee, comprising specialists in key disciplines from the departments of Medicine and Paediatrics, was formed. We utilized the IVLE, developed for our university in 1998, as the online platform on which we ran the MEQ. We calculated the number of man-hours (academic and support staff) required to run the MEQ examination, using either a computer-based or pen-and-paper format.Results: With the support of our university's information technology (IT) specialists, we have successfully run the MEQ examination online, twice a year, since 2003. Initially, we conducted the examination with short-answer questions only, but have since expanded the MEQ examination to include multiple-choice and extended matching questions. A total of 1268 man-hours was spent in preparing for, and running, the MEQ examination using CBT, compared to 236.5 man-hours to run it using a pen-and-paper format. Despite being more labour-intensive, our students and staff prefer CBT to the pen-and-paper format.Conclusions: The MEQ can be conducted using a computer-based testing scenario, which offers several advantages over a pen-and-paper format. We hope to increase the number of questions and incorporate audio and video files, featuring clinical vignettes, to the MEQ examination in the near future.

EMQS – how to write and how to answer

The shift to extended matching questions means that some people setting or sitting college examinations will be faced with an unfamiliar assessment method. This article briefly outlines the process of examination construction, the advantages and disadvantages of extended matching questions and an approach to such questions when writing and answering examination papers.

Diagnostic ability in relation to clinical seminars and extended-matching questions examinations

At the Katholieke Universiteit Leuven, Belgium, final year medical students participate in 70 problem-solving clinical seminars over a period of 2 months, concentrating on clinical reasoning for formulating differential diagnoses. The clinical seminars end in an examination consisting of 200 extended-matching questions (EMQs). This study asks whether problem-solving clinical seminars improve clinical reasoning and whether the EMQ examination measures clinical decision making. In order to detect an increase in diagnostic ability, the Diagnostic Thinking Inventory (DTI) was applied. The research hypotheses were: (i) DTI scores will be higher after the seminars than before, and (ii) the correlations between DTI scores and EMQ examination scores will be significant. In the academic year 2003-04, 3 series of problem-solving clinical seminars were held. At the beginning and end of each series the students filled in the DTI. This questionnaire measures 2 aspects of diagnostic thinking: the degree of flexibility in thinking, and how knowledge is structured in the memory. For all data together, the DTI scores after the clinical seminars were significantly higher than before. Pearson correlations between DTI scores and EMQ examination scores were low but significant, with the exception of post-test Structure (not significant). Two months of intensive problem-solving clinical training was accompanied by an improvement in diagnostic thinking, as measured by DTI scores. Correlation between DTI scores and examination scores indicates that the EMQs measure an aspect of student achievement that is related to clinical reasoning.