Critical Appraisal Of Medical Literature Research Articles

The Design and Validation of a Self-Assessment Questionnaire of Critical Appraisal of Medical Literature Skills in Medical Students.

Critical appraisal of medical literature is a challenging step of the evidence-based medicine practice. Many assessment questionnaires have been published in the literature, but they have mainly focused on all the evidence-based medicine practice process. The authors aimed to develop and validate a questionnaire assessing the critical appraisal skills of medical students from the same Faculty. The questionnaire was developed by item generation through a review of the literature and an expert committee. The questionnaire was validated in terms of content validity and construct validity. Fitness of data for analysis was checked through Kaiser-Meyer-Olkin (KMO) and Bartlett's sphericity. Construct validity was carried out using a principal axis exploratory factor analysis (EFA) with 'varimax' rotation to study the internal structure of the questionnaire and to extract the test major factors. The questionnaire was administrated to a cohort of under and postgraduate medical students (n=84) to evaluate the test reliability and select the best items. The reliability of the questionnaire was assessed by Cronbach's alpha coefficient to evaluate the internal consistency. The correlations between the self-confidence and satisfaction dimension score, the critical thinking dimension score, the learning style dimension score, the Fresno-adapted test scores and the total score were assessed using the Spearman's correlation test. The questionnaire consisted of 31 items. A factorial analysis grouped the items into 3 dimensions that consisted of the self-confidence and satisfaction dimension, the critical thinking dimension and the learning style dimension. Cronbach's alpha accounted for 0.95, CI95% [0.9-1] for the entire questionnaire. The factor analysis explained 79.51% of the variance. The external validity assessment based on a Spearman's correlation study highlighted a weak correlation between the total scores and the critical thinking dimension and the self-perception and satisfaction dimension. In spite of the limitations of this study, mainly the small number of the students recruited, the questionnaire seems to measure with adequate reliability the competences of under and postgraduate medical students.

Critical appraisal of medical literature in undergraduate and postgraduate medical students

BackgroundCritical appraisal of medical literature is a challenging competency which is mandatory in the practice of evidence-based medicine. Achieving this competency has been reported to vary according to many factors including the students’ background, gender, or specialty. The majority of the articles published in this field mainly assessed the students’ satisfaction or their cognitive knowledge. The authors aimed to assess the possibility of performing the same training of critical appraisal of medical literature to students with different backgrounds to highlight the link between the critical appraisal practice and the reflexive, cognitive, attitudes and conative profile of the students according to their level.MethodsStudents in the second year (SYME) and third year (TYME) of medical education and students performing continuing medical education (CME) and family doctors (FD) were invited to voluntarily participate in the training. This study occurred from September 2020 to June 2022. Before the workshop, the students were invited to answer a self-assessment questionnaire and a prerequisite test. Two original manuscripts dealing with COVID-19 were analyzed using published checklists. At the end of the workshops, all of the students fulfilled a second self-assessment questionnaire and a Fresno-adapted final test assessing their knowledge. Both self-assessment questionnaires were conceived by an expert committee according to the different steps of questionnaires’ validation. They assessed the cognitive, conative, motivational, and reflexive students’ profile using Likert scale questions. Also, the participants were invited to answer a semi-structured interview.ResultsNinety-five participants were included with 6 participants in SYME, 54 in TYME, 16 in CME, and 19 FD. The results highlighted the possibility of using the same program independently from the students’ level when using manuscripts dealing with a widely known and confusing disease such as COVID-19. The analysis of the different pre- and postworkshop questionnaire scores highlighted mild or negative scores in the TYME and intermediate or good scores in the other levels. The variation of the mean scores according to the students’ level revealed a significant difference in prerequisite, final test, pre and postworkshop attitudes, postworkshop cognitive, postworkshop conative and satisfaction scores. 11 students from different levels were interviewed. The content analysis highlighted 4 themes including the teaching organization, the institutional assessment, the impact on the research practice and the impact on the patients’ management with the emphasis of the TYME students on the necessity of integrating the teaching and the assessment of the EBM principles in the curriculum.ConclusionOur results put emphasis on the necessity of introducing the teaching of critical appraisal of medical literature early in the curriculum in order to avoid negative behaviorism during the clerkship period.

An Instrument for Measuring Critical Appraisal Self-Efficacy in Rheumatology Trainees.

Self-efficacy, the internal belief that one can perform a specific task successfully, influences behavior. To promote critical appraisal of medical literature, rheumatology training programs should foster both competence and self-efficacy for critical appraisal. This study aimed to investigate whether select items from the Clinical Research Appraisal Inventory (CRAI), an instrument measuring clinical research self-efficacy, could be used to measure critical appraisal self-efficacy (CASE). One hundred twenty-five trainees from 33 rheumatology programs were sent a questionnaire that included two sections of the CRAI. Six CRAI items relevant to CASE were identified a priori; responses generated a CASE score (total score range 0-10; higher=greater confidence in one's ability to perform a specific task successfully). CASE scores' internal structure and relation to domain-concordant variables were analyzed. Questionnaires were completed by 112 of 125 (89.6%) trainees. CASE scores ranged from 0.5 to 8.2. The six CRAI items contributing to the CASE score demonstrated high internal consistency (Cronbach's α=0.95) and unidimensionality. Criterion validity was supported by the findings that participants with higher CASE scores rated their epidemiology and biostatistics understanding higher than that of peers (P < 0.0001) and were more likely to report referring to studies to answer clinical questions (odds ratio 2.47, 95% confidence interval 1.41-4.33; P=0.002). The correlation of CASE scores with percentage of questions answered correctly was only moderate, supporting discriminant validity. The six-item CASE instrument demonstrated content validity, internal consistency, discriminative capability, and criterion validity, including correlation with self-reported behavior, supporting its potential as a useful measure of critical appraisal self-efficacy.

The Hidden Curriculum of the COVID-19 Pandemic.

The Hidden Curriculum of the COVID-19 Pandemic.

Using a Journal Club Series to Introduce Paramedic Students to Research Fundamentals and Critical Appraisal of Medical Literature.

Despite United States national learning objectives referencing research fundamentals and the critical appraisal of medical literature, many paramedic programs are not meeting these objectives with substantive content. The objective was to develop and implement a journal club educational module for paramedic training programs, which is all-inclusive and could be distributed to Emergency Medical Services (EMS) educators and EMS medical directors to use as a framework to adapt to their program. Four two-hour long journal club sessions were designed. First, the educator provided students with four types of articles on a student-chosen topic and discussed differences in methodology and structures. Next, after a lecture about peer-review, students used search engines to verify references of a trade magazine article. Third, the educator gave a statistics lecture and critiqued the results section of several articles found by students on a topic. Finally, students found an article on a topic of personal interest and presented it to their classmates, as if telling their paramedic partner about it at work. Before and after the series, students from two cohorts (2017, 2018) completed a survey with questions about demographics and perceptions of research. Students from one cohort (2017) received a follow-up survey one year later. For the 2016 cohort, 13 students participated and provided qualitative feedback. For the 2017 and 2018 cohorts, 33 students participated. After the series, there was an increased self-reported ability to find, evaluate, and apply medical research articles, as well as overall positive trending opinions of participating in and the importance of prehospital research. This ability was demonstrated by every student during the final journal club session. McNemar's and Related-Samples Cochran's Q testing of questionnaire responses suggested a statistically significant improvement in student approval of exceptions from informed consent. The framework for this paramedic journal club series could be adapted by EMS educators and medical directors to enable paramedics to search for, critically appraise, and discuss the findings of medical literature.

#GeriMedJC: The Twitter Complement to the Traditional-Format Geriatric Medicine Journal Club.

Twitter is a public microblogging platform that overcomes physical limitations and allows unrestricted participation beyond academic silos, enabling interactive discussions. Twitter-based journal clubs have demonstrated growth, sustainability, and worldwide communication, using a hashtag (#) to follow participation. This article describes the first year of #GeriMedJC, a monthly 1-hour live, 23-hour asynchronous Twitter-based complement to the traditional-format geriatric medicine journal club. The Twitter moderator tweets from the handle @GeriMedJC; encourages use of #GeriMedJC; and invites content experts, study authors, and followers to participate in critical appraisal of medical literature. Using the hashtag #GeriMedJC, tweets were categorized according to thematic content, relevance to the journal club, and authorship. Third-party analytical tools Symplur and Twitter Analytics were used for growth and effect metrics (number of followers, participants, tweets, retweets, replies, impressions). Qualitative analysis of follower and participant profiles was used to establish country of origin and occupation. A semistructured interview of postgraduate trainees was conducted to ascertain qualitative aspects of the experience. In the first year, @GeriMedJC has grown to 541 followers on six continents. Most followers were physicians (43%), two-thirds of which were geriatricians. Growth metrics increased over 12months, with a mean of 121 tweets, 25 participants, and 105,831 impressions per journal club. Tweets were most often related to the article being appraised (87.5%) and ranged in thematic content from clinical practice (29%) to critical appraisal (24%) to medical education (20%). #GeriMedJC is a feasible example of using social media platforms such as Twitter to encourage international and interprofessional appraisal of medical literature.

Characteristics of evidence-based medicine training in Royal College of Physicians and Surgeons of Canada emergency medicine residencies - a national survey of program directors.

BackgroundRecent surveys suggest few emergency medicine (EM) training programs have formal evidence-based medicine (EBM) or journal club curricula. Our primary objective was to describe the methods of EBM training in Royal College of Physicians and Surgeons of Canada (RCPSC) EM residencies. Secondary objectives were to explore attitudes regarding current educational practices including e-learning, investigate barriers to journal club and EBM education, and assess the desire for national collaboration.MethodsA 16-question survey containing binary, open-ended, and 5-pt Likert scale questions was distributed to the 14 RCPSC-EM program directors. Proportions of respondents (%), median, and IQR are reported.ResultsThe response rate was 93% (13/14). Most programs (85%) had established EBM curricula. Curricula content was delivered most frequently via journal club, with 62% of programs having 10 or more sessions annually. Less than half of journal clubs (46%) were led consistently by EBM experts. Four programs did not use a critical appraisal tool in their sessions (31%). Additional teaching formats included didactic and small group sessions, self-directed e-learning, EBM workshops, and library tutorials. 54% of programs operated educational websites with EBM resources. Program directors attributed highest importance to two core goals in EBM training curricula: critical appraisal of medical literature, and application of literature to patient care (85% rating 5 - “most importance”, respectively). Podcasts, blogs, and online journal clubs were valued for EBM teaching roles including creating exposure to literature (4, IQR 1.5) and linking literature to clinical practice experience (4, IQR 1.5) (1-no merit, 5-strong merit). Five of thirteen respondents rated lack of expert leadership and trained faculty educators as potential limitations to EBM education. The majority of respondents supported the creation of a national unified EBM educational resource (4, IQR 1) (1-no support, 5- strongly support).ConclusionsRCPSC-EM programs have established EBM teaching curricula and deliver content most frequently via journal club. A lack of EBM expert educators may limit content delivery at certain sites. Program directors supported the nationalization of EBM educational resources. A growing usage of electronic resources may represent an avenue to link national EBM educational expertise, facilitating future collaborative educational efforts.

The teaching of critical appraisal skills in urology residency: What is the evidence?

The manuscript by Roth and Siemens1 is a topical one that deals with attitudes towards and knowledge of evidence-based medicine (EBM) among graduating urology residents. The authors also discuss the prevalence and structure of EBM curricula in Canadian urology training programs. The findings from this self-reported survey of 29 chief residents are concerning but not surprising. Formal teaching of EBM in residency was reported by less than a third of respondents and only slightly more than half received instruction during medical school. Most urology resident exposure to EBM occurs in a journal club format and most respondents were not aware of 8 commonly used web-based EBM resources available to practitioners. Knowledge of the fundamentals of EBM terminology was correspondingly limited, but there was strong support for developing formal instruction and evaluation of these important concepts. A lack of time during residency and a lack of faculty with the requisite expertise were cited as the most common reasons for the limited exposure to EBM. These are similar to barriers reported in other specialty training programs as outlined in a recent systematic review of this topic.2 In general, limited time, competing responsibilities and resident lack of knowledge and skills were cited as the most common barriers to the application of EBM in the published literature. Interestingly, in one Canadian study included in this review, several surgical residents reported a fear of repercussion from staff surgeons when confronting them with new evidence.3 (There were no urology residents in this study!) As new faculty with advanced education in EBM come on board, our training of urology residents in this area may self-correct as long as the teachers are enabled to apportion the necessary time in urology curricula to this important facet of postgraduate education. This is easier said than done. There is considerable curriculum compression in residency training programs today with requirements for teaching of all the CanMEDS core competencies to maintain Royal College accreditation. Evidence-based medicine falls under the scholar rubric of CanMEDS. Item 6.2 of the Royal College general standards of accreditation states that “The program must ensure that there are effective teaching programs in the critical appraisal of medical literature using knowledge of research methodology and biostatistics.”4 Hosting journal clubs as the sole method of teaching critical appraisal is not likely to be deemed sufficient to receive a passing grade by an external reviewer of our residency programs. Under the old apprenticeship model of training, scholarship involved honing our clinical skills in concert with the incorporation of expert opinion into one’s personal database. In the hierarchy of EBM, expert opinion is the lowest level of evidence and therefore a lower value is placed on authority than did previous traditional medical paradigms.5,6 Urologists need to continue to climb the pyramidal levels of evidence if we are to remain a scholarly specialty. That said, not all urologists (me included) need to be the principal investigator of a randomized controlled trial, be capable of conducting a meta-analysis, or wax eloquent about the difference between an odds ratio, a risk ratio, and a hazard ratio. However, we should all be aware of what these tools entail, whether they are in existence for questions that arise in our clinical practices, and where to access them if they do exist.

Calculation skills for nurses: epidemiological measures

T topic of this article in the calculation skills series is the calculation of epidemiological measures from clinical trial data. The article will not discuss the relative merits of different kinds of clinical trial; instead, an excellent series of tools supporting the critical appraisal of medical literature is available from the Centre for Evidence Based Medicine (CEBM, 2011) The clinical trials selected to illustrate each calculation were chosen at random and have not been critically appraised by the author. We are simply going to look at how to calculate some of the measures, and what they mean in practice. The measures we will look at are: relative risk, relative risk reduction and absolute risk reduction. Relative risk is the ratio of the probability of an event occurring in an exposed group compared with the probability of it occurring in a non-exposed group. It can be used to analyse clinical trial data both for benefits and risks of treatment or to describe the risk of an event occurring. If the relative risk is 1, this means there is no difference in risk between the experimental and control groups. Relative risk reduction is the percentage reduction in risk between the experimental and control groups. Absolute risk reduction is the difference between the event rate in an experimental group receiving a particular treatment compared with the event rate in the control group. The control group could receive either placebo or a different drug. The equation to calculate relative risk is:

Statistics in Medicine

The scope of biomedical research has expanded rapidly during the past several decades, and statistical analysis has become increasingly necessary to understand the meaning of large and diverse quantities of raw data. As such, a familiarity with this lexicon is essential for critical appraisal of medical literature. This article attempts to provide a practical overview of medical statistics, with an emphasis on the selection, application, and interpretation of specific tests. This includes a brief review of statistical theory and its nomenclature, particularly with regard to the classification of variables. A discussion of descriptive methods for data presentation is then provided, followed by an overview of statistical inference and significance analysis, and detailed treatment of specific statistical tests and guidelines for their interpretation.

Influence de la lecture critique d’articles sur les classements des étudiants aux épreuves classantes nationales 2009

To assess the impact of the introduction of the critical appraisal of medical literature (LCA) on the rating ranks of students and faculties of medicine in the National Ranking (NR). A secondary objective was the evaluation of each test of the NR on the rating ranks of students and faculties of medicine. From the official results of the NR in 2009, the academic ranks of students were recalculated after eliminating the note in each file. Medical schools were ranked on the basis of mean scores for students from these schools in each file. The notes of 6258 students tested in 40 faculties of medicine and the military health department were selected. The Spearman rank correlation for LCA notes and clinical records was Rho=0.44 (p<0.001). The test of LCA aims to award 561 places to a student and lose up to another 595. The rate ranking rank of a faculty of medicine has been modified by 5 places with the LCA. The LCA test in 2009 had a significant effect on the rate ranking of students, whereas the LCA accounted for only 5 % of the total score. The LCA test effect on the ranking of faculties of medicine was more modest. The influence of the LCA may be more pronounced in 2010 because it will count for 10 % of the notes. The rate ranks were heterogeneous within each medical faculty, which should raise questions about the teaching of some medical specialities in each faculty.

Fiabilité et validité d'une épreuve de lecture critique d'articles médicaux scientifiques

The aim of this study was to assess the reliability and validity of tests assessing medical students' ability to read medical journal articles critically.All third-year students at our medical school (n=125) took three tests between December 2004 and June 2005 as part of a program to teach critical appraisal of medical literature. Each test required students to write a 250-word structured abstract and to answer a set of 6 to 8 appraisal questions. Interrater reliability was assessed with the intra-class correlation coefficient for the first test. Validity was assessed 1) by calculating the correlation coefficient between the mean scores on the three critical appraisal tests and 2) by calculating the correlation coefficient between these tests and eight other tests in other subjects that the students took over the same period.Interrater reliability was satisfactory for the overall score (ICC=0.72) and for the score for the questions alone (ICC=0.73). It was only moderate, however, for the structured abstract test (ICC=0.53) and varied markedly from question to question (ICC range: 0.29-0.86). Intertest correlations were all statistically significant (p<0.001) and the mean score on the critical reading tests was significantly correlated with the mean score on the tests for other subjects that year (r=0.60; p<0.001).The validity and reliability of the tests of critical reading of medical literature are satisfactory but can be improved by developing more explicit scoring templates.

Increasing Resident Research in a Military Internal Medicine Program

The internal medicine residents from our training program rarely participated in scholarly activity prior to 1994. In an attempt to increase the quality and quantity of resident research, we initiated a research program that included: (1) a mandatory research project, (2) 2 months of dedicated research time, (3) appointment of a research director, (4) lectures on the critical appraisal of medical literature and research design, (5) technical support, (6) faculty mentoring, (7) a research meeting and competition involving five residency programs, and (8) military achievement awards. From 1994 to 1999, our house staff has had 134 research presentations and 21 manuscripts accepted for publication in peer-reviewed journals. Ninety percent of our residents presented at least one project at a scientific meeting by completion of their training between 1996 and 1999. Resident scholarly activity is significantly enhanced by a structured research program, an opportunity to present at a scientific meeting, and award recognition.

Critical appraisal of medical literature: an aid to rational decision making.

Clinical decisions should be based on evidence. This paper explores the skills necessary to read critically and evaluate medical literature, and in particular discusses how to appraise a study from a general practice perspective. It outlines the different types of study used in medical literature and the weight that should be attributed to each. The anatomy of a research paper is described indicating the important features to be found in each section of a study; the Introduction, Methods, Results, Discussion and Conclusion (IMRAD) and the essential components, possible problems, and sources of bias that may be found in each section. The need for close examination of the methodology is emphasised, as this determines the quality of the study, with sufficient detail provided in the methods section that the study could be replicated. Sampling and bias are explored in detail and the important features of a measuring instrument are described. Response rates, confounding factors and the use of statistics are also highlighted. The study upon which the research paper is based is usually the best that was possible within resource limitations and, while the evidence should be appraised objectively, the quality of the study should be seen within the context of these limitations.

154 EFFECT OF STRUCTURED ABSTRACTS (SA) ON ABSTRACT QUALITY

Hypothesis: SA contain more information than unstructured. Study design: A comparison of original abstracts (OA) and resubmitted SA. Setting: Pairs of OA and resubmitted SA to the ESPR, 1993 “Subjects”: Following receipt of OA, authors were asked to resubmit using structured format suggested by the Working Group on Critical Appraisal of Medical Literature, Ann Intern Med.106:598-604. 72 were resubmitted on the same abstract form (SA form), 94 on one side of A4 (SA A4) - as requested. Measurements: Each reviewer (2 ex presidents ESPR) assessed 1 abstract from each pair giving a Scientific score out of 6 and a time for review. The information content (IC) was then objectively measured by third ex president. Conclusion: The abstracts were perceived better and contained more information when submitted in a structured format. The IC can be increased very significantly without necessarily increasing abstract size or review time.