Abstract
BackgroundKnowledge assessment in evidence-based medicine (EBM) is usually performed by the measurement of memorised facts, understanding of EBM concepts and application of learned knowledge in familiar situations, all of which are considered lower-level educational objectives. The aim of this study was to assess EBM knowledge both on higher and lower cognitive levels across EBM topics.MethodsIn order to assess knowledge on different EBM topics across learning levels, we created a knowledge test (Six Progressive Levels in Testing – SPLIT instrument), which consists of 36 multiple choice items and measures knowledge in EBM at six cognitive levels (Remembering, Understanding, Applying, Analysing, Evaluating and Creating) and addresses six EBM topics (Evidence-based practice, Internal validity, Clinical importance, Study design, Sources of evidence, Diagnostic studies). Three independent assessors defined the minimum passing score (MPS) for the overall test, based on the first-year course content and educational objectives. The instrument was assessed in a sample of first- (n = 119) and third-year medical students (n = 70) and EBM experts (n = 14).ResultsThe MPS was 16 correct answers out of total 36 questions, and was achieved by 21 out of 119 first-year students, 14 out of 70 third-year students and 9 out of 14 EBM experts (χ2 = 13.3; P < 0.001, with significantly higher proportion of experts passing compared to students). Although experts had the highest scores overall, none of the groups outperformed others on individual cognitive levels, but the experts outperformed students in EBM topics of Study design and Sources of evidence (P = 0.002 and 0.004, respectively, Kruskal-Wallis test). First- and third-year students performed better on specific course topics taught in that study year (Diagnostic studies and Clinical relevance, respectively).ConclusionEBM knowledge of students and experts differ according to the specificities of their education/expertise, but neither group had excellent knowledge in all areas. It may be difficult to develop a knowledge test that includes different EBM topics at different cognitive levels to follow the development of specific and general aspects of EBM knowledge.
Highlights
Knowledge assessment in evidence-based medicine (EBM) is usually performed by the measurement of memorised facts, understanding of EBM concepts and application of learned knowledge in familiar situations, all of which are considered lower-level educational objectives
EBM is often emphasized as a skill that needs to be transferred to everyday practice, it is not clear how EBM training leads to the educational objectives developed in theoretical frameworks [6]
We calculated the discrimination indexes by comparing the data from the top performing 25% and bottom performing 25% of the participants, first for their overall scores on the test, and for the scores on individual cognitive levels and Results In the pilot study, no difference was found between the groups that took the test with open- ended answers (Md = 19, 95% confidence intervals (CIs) = 16 to 19, n = 46) and those that took the test with multiple choice answers (Md = 20, 95% CI = 18 to 21, n = 42; P = 0.722)
Summary
Knowledge assessment in evidence-based medicine (EBM) is usually performed by the measurement of memorised facts, understanding of EBM concepts and application of learned knowledge in familiar situations, all of which are considered lower-level educational objectives. Curriculum designers are encouraged to express their educational objectives according to students’ abilities and competencies [6,7,8] Those educational objectives, sometimes differently defined, create sharp differences between the simple memorisation of the material or superficial overview of the information and critical assessment of the acquired information. We compared the three most widely used measures of EBM: the ACE tool [11], Fresno test [12] and Berlin questionnaire [13] These tests differ in question type, scoring and focus on EBM topics, rendering the results of the student groups or the educational intervention dependent on the choice of the EBM test [14]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
