Searching for evidence-based medicine in the literature. Part 3: Assessment.

Abstract

After composing a pointed question for an evidence-based medicine (EBM) search and then selecting appropriate resources for its execution, the final step requires an assessment of the information gleaned: what is substantive material and what is not? Unless one is especially adept at statistics and/or EBM methodology, it may appear that one study is as good as another and, consequently, the researcher can feel muddled by the retrieval. Critical appraisal in EBM generally centers around three questions: 1) is it valid?, 2) what are the results?, and 3) is it applicable to the patient? One is trying to determine if the study’s evidence demonstrates enough rigor within its methodology to impact the course of a particular patient’s care. It may be tempting to view critical appraisal as an invitation to negate or castigate clinical study efforts but, in a real world, no study can be perfectly designed or executed. The main purpose of critical appraisal is to “assess not whether a paper is ‘rubbish’, but whether there is so much potential bias that the results are no longer valid”.1 While the determination of validity is not easy and relevance to patient care is sometimes obscured, two resources that can assist in the evaluation of good, better, and best EBM information can be accessed electronically. At the Critical Appraisal Skills Programme (CASP) section of the United Kingdom’s National Health Service web site (http://www.phru.nhs.uk/casp/appraisa.htm), a wealth of resources provides a rich repository. Especially useful are forms described as critical appraisal tools for various types of articles including systematic reviews, randomized controlled trials, cohort studies, case control studies, diagnostic test studies and economic evaluation studies. Each form consists of ten questions about each specific type of article, which are supplemented with prompts that assist in the assessment of each question topic. The questions derive from a notable series titled “Users’ Guides to the Medical Literature,” which appeared in the Journal of the American Medical Association (JAMA) from 1993 through 20002 – 34 and, as such, constitute a solid approach to evidence-based assessment. In terms of function, the forms demonstrate attention to clarity in font size, strong contrast in design, and an efficient screening section in each article type that determines whether one should proceed through all ten questions or stop. While the roomy, clear forms may be used only for an individual’s personal use, the web site also provides a method to obtaining copyright permission if one wanted to use them in a group practice or other setting. Another useful web site that offers a step-by-step approach to critical appraisal can be found at the Centre for Evidence-Based Medicine in Oxford, England (http://www.cebm.net/critical_appraisal.asp). Worksheets provided for the assessment of articles relating to diagnosis, harm, etiology, therapy and systematic reviews of therapy can be accessed separately or downloaded as a group. Each type of article evaluation requires a different approach to determination of validity; critical questions and demonstrations of specific calculations are customized for each worksheet set. The addition of statistics can be daunting, but the authors supply understandable sample calculations and then provide a practice section for the user’s skill development. The last section of the form consists of patient-related questions to focus the results of the inquiry and, as such, cinches the usefulness of the worksheets. The aim of the worksheet exercise is to determine the article’s relevance to an individual patient’s care, and these worksheets provide an effective and cogent methodology to do so. While there are many evidence-based medicine resources, it is more difficult to find those with an easy and efficient approach to a rather complex area. Without compromising quality or exhausting time, one can find respectable EBM web resources to compose a suitable question, execute a viable literature search, and then assess the trustworthiness of the materials gleaned. Certainly more time and resources are needed for extended systematic assessments, but the busy practitioner can use evidence-based medicine in daily practice and still feel comfortable about the results for specific patient care.