GRADE Quality Of Evidence Research Articles

The effectiveness of sub-group specific manual therapy for low back pain: A systematic review

BackgroundManual therapy is frequently used to treat low back pain (LBP), but evidence of its effectiveness is limited. One explanation may be sample heterogeneity and inadequate sub-grouping of participants in randomized controlled trials (RCTs) where manual therapy has not been targeted toward those likely to respond. ObjectivesTo determine the effectiveness of specific manual therapy provided to sub-groups of participants identified as likely to respond to manual therapy. Data sourcesA systematic search of electronic databases of MEDLINE, EMBASE, CINAHL, and the Cochrane Central Register of Controlled trials (CENTRAL). Trial eligibility criteriaRCTs on manual therapy for participants identified as belonging to a sub-group of LBP likely to respond to manual therapy were included. Trial appraisal and synthesis methodsIdentified trials were assessed for eligibility. Data from included trials were extracted by two authors independently. Risk of bias in each trial was assessed using the PEDro scale and the overall quality of evidence rated according to the GRADE domains. Treatment effect sizes and 95% confidence intervals were calculated for pain and activity. ResultsSeven RCTs were included in the review. Clinical and statistical heterogeneity precluded meta-analysis. Significant treatment effects were found favouring sub-group specific manual therapy over a number of comparison treatments for pain and activity at short and intermediate follow-up. However, the overall GRADE quality of evidence was very low. ConclusionsThis review found preliminary evidence supporting the effectiveness of sub-group specific manual therapy. Further high quality research on LBP sub-groups is required.

Managerial supervision to improve primary health care in low- and middle-income countries.

Primary healthcare (PHC) workers often work alone or in isolation. Healthcare managerial supervision is recommended to help assure quality; but this requires skilled supervisors and takes time and resources. It is therefore important to assess to what extent supervision is beneficial and the ways in which it can be implemented. To review the effects of managerial supervision of health workers to improve the quality of PHC (such as adherence to guidance or coverage of services) in low- and middle-income countries. We searched The Cochrane Central Register of Controlled Trials (CENTRAL) 2011, Issue 1, part of The Cochrane Library. www.thecochranelibrary.com, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 10 March 2011); MEDLINE, Ovid 1950 to March Week 1 2011 (searched 08 March 2011); EMBASE, Ovid 1980 to 2011 Week 12 (searched 08 March 2011); CINAHL, Ebsco 1981 - present (searched 10 March 2011); LILACS, VHL (searched 10 March 2011). Randomised controlled trials, controlled before-and-after studies, and interrupted time series studies, conducted in PHC in low- and middle-income countries. Supervision includes site visits from a central level of the health system, plus at least one supervisory activity. We excluded studies aimed solely at improving the clinical skills of PHC workers. We extracted data using a predefined form and assessed for risk of bias using the EPOC risk of bias criteria. Data are presented in a narrative way without pooling the effects on the outcomes as studies and outcomes were diverse. Nine studies met the inclusion criteria: three compared supervision with no supervision, five compared enhanced supervision with routine supervision, and one study compared less intensive supervision with routine supervision. Most outcomes were scores relating to providers' practice, knowledge and provider or user satisfaction. The majority of the outcomes were measured within nine months after the interventions were introduced. In two studies comparing supervision with no supervision, small benefits on provider practice and knowledge were found. For methods of enhancing supervision, we identified five studies, and two studies of frequent supportive supervision demonstrated small benefits on workers performance. The one study examining the impact of less intensive supervision found no evidence that reducing the frequency of visits had any effect on the utilisation of services. The GRADE evidence quality for all comparisons and outcomes was "low" or "very low". It is uncertain whether supervision has a substantive, positive effect on the quality of primary health care in low- and middle-income countries. The long term effectiveness of supervision is unknown.

Male circumcision for prevention of homosexual acquisition of HIV in men

Previous systematic reviews found inconsistent effects of male circumcision on HIV acquisition in men who have sex with men (MSM). However, a number of new studies have become available in the three years since the last systematic review. To assess the effects of male circumcision for preventing HIV acquisition by men through sex with men. In June 2010 we electronically searched the Cochrane Central Register of Controlled Trials, PubMed, EMBASE, AIDS Education Global Information System, ClinicalTrials.gov, and WHO International Clinical Trials Registry Platform; hand-searched reference lists of relevant articles; and contacted relevant organisations and experts. We updated the search in March 2011. We looked for randomised controlled trials (RCTs) and observational studies that assessed the effects of male circumcision on HIV acquisition in MSM. Two authors independently assessed study eligibility and methodological quality, and extracted data. We expressed study results as odds ratios (OR) with 95% confidence intervals (CI), and conducted random-effects meta-analysis. We found no completed RCT and included 21 observational studies with 71,693 participants. The only eligible RCT is currently ongoing among MSM in China. The pooled effect estimate for HIV acquisition was not statistically significant (20 studies; 65,784 participants; OR 0.86, 95% CI 0.70 to 1.06) and showed significant heterogeneity (I²=53%). In a subgroup analysis, the results were statistically significant in studies of men reporting an insertive role (7 studies, 3465 participants; OR 0.27, 95% CI 0.17 to 0.44; I²=0%) but not in studies of men reporting a receptive role (3 studies, 1792 participants; OR 1.20, 95% CI 0.63 to 2.29; I² = 0%). There was no significant association between male circumcision and syphilis (8 studies; 34,999 participants: OR 0.96, 95% CI 0.82 to 1.13; I² = 0%), herpes simplex virus 1 (2 studies, 2740 participants; OR 0.90, 95% CI 0.53 to 1.52; I²=0%), or herpes simplex virus 2 (5 studies;10,285 participants; OR 0.86, 95% CI 0.62 to 1.21; I²=0%). The overall GRADE quality of evidence was low. None of the included studies assessed adverse effects associated with male circumcision. Current evidence suggests that male circumcision may be protective among MSM who practice primarily insertive anal sex, but the role of male circumcision overall in the prevention of HIV and other sexually transmitted infections among MSM remains to be determined. Therefore, there is not enough evidence to recommend male circumcision for HIV prevention among MSM at present. Further research should be of high quality and further explore interaction with the predominant sexual role.

Grading quality of evidence and strength of recommendations in clinical practice guidelines Part 3 of 3. The GRADE approach to developing recommendations

This is the third and last article in the series about the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the quality of evidence and the strength of recommendations in clinical practice guidelines and its application in the field of allergy. We describe the factors that influence the strength of recommendations about the use of diagnostic, preventive and therapeutic interventions: the balance of desirable and undesirable consequences, the quality of a body of evidence related to a decision, patients' values and preferences, and considerations of resource use. We provide examples from two recently developed guidelines in the field of allergy that applied the GRADE approach. The main advantages of this approach are the focus on patient important outcomes, explicit consideration of patients' values and preferences, the systematic approach to collecting the evidence, the clear separation of the concepts of quality of evidence and strength of recommendations, and transparent reporting of the decision process. The focus on transparency facilitates understanding and implementation and should empower patients, clinicians and other health care professionals to make informed choices.

Clinical guidelines for testing for heritable thrombophilia

Trevor Baglin, Elaine Gray, Mike Greaves, Beverley J. Hunt, David Keeling, Sam Machin, Ian Mackie, Mike Makris, Tim Nokes, David Perry, R. C. Tait, Isobel Walker and Henry Watson Addenbrooke’s Hospital, Cambridge, NIBSC, South Mimms, University of Aberdeen, Aberdeen, Guy’s and St Thomas’, London, Churchill Hospital, Oxford, University College Hospital, London, Royal Hallamshire Hospital, Sheffield, Derriford Hospital, Plymouth, Glasgow Royal Infirmary, Glasgow and Aberdeen Royal Infirmary, UK

Reporting of Harm in Randomized Controlled Trials Published in the Urological Literature

Evidence-based decision making seeks to balance potential benefits and harms (adverse effects) of health care interventions for an individual patient. We determined the prevalence and completeness of harm reporting in randomized controlled trials in the urological literature. We performed a systematic literature search of all randomized controlled trials of therapeutic interventions published in The Journal of Urology, Urology, European Urology and BJU International in 1996 and 2004. Each article was reviewed by 2 independent investigators for 10 harm reporting criteria recommended by the CONSORT group. Discrepancies were settled by discussion and consensus. A total of 152 randomized controlled trials met the inclusion criteria, of which 109 (72%) reported adverse event outcomes. The median number of harm reporting criteria satisfied improved marginally from 1996 to 2004 (2.8 to 3.3, p = 0.36). A large proportion of studies failed to address harm in the abstract (55, 36%), introduction (71, 47%) and discussion (52, 34%). Few studies specified which adverse events were evaluated (21, 14%), when harm information was collected (32, 21%) or how the harm was attributed to the intervention (5, 3%). Only 48 (32%) articles provided reasons for patient withdrawal and 1 in 5 (33, 22%) reported the severity of adverse events. Randomized controlled trials published in the urological literature contain significant deficiencies in adverse event reporting. These findings suggest the need for reporting standards for harm in urological journals. Improvements in adverse event reporting would permit a more balanced assessment of interventions and would enhance evidence-based urological practice.

Grading Quality of Evidence and Strength of Recommendations: A Perspective

Evidence-based practice requires translating research findings into clinical and policy decision making. Clinical practice guidelines (CPGs) serve this purpose by evaluating evidence and making recommendations about therapeutic and diagnostic interventions and clinical management strategies. Systematic reviews are considered the best available evidence and are often used in the development of CPGs [1,2]. Since guideline development involves an assessment of the overall quality of evidence and complex balancing of trade-offs between the important benefits and harms of any given intervention, arbitrariness, value judgements, and subjectivity ultimately come into play in the guideline development process and associated recommendations [3]. In order to minimize cognitive bias in interpreting evidence and make the inherently subjective process more transparent and consistent, CPGs have traditionally employed formal systems or frameworks to understand and grade the quality of the body of evidence and strength of recommendations [4,5]. One such framework is the grading quality of evidence and strength of recommendations (GRADE), which is commonly used by guideline panels in deriving health care recommendations. GRADE was developed to overcome some of the deficiencies of earlier efforts [6]. GRADE defines the quality of evidence as the collective level of confidence guideline developers have about the validity of estimates of benefits and harms for any given intervention, and the strength of guideline recommendation as the extent of collective confidence that adherence to the recommendation will do more good than harm [7]. It urges guideline developers to consider all important patient outcomes of benefit and harm, to systematically evaluate the quality of their estimates, and to assess the trade-offs between evidence of benefits and harms, the preferences and values placed by patients on outcomes, the opportunity cost associated with the recommendation, and the feasibility of recommendations given a clinical setting before formulating guideline recommendations. Details of the GRADE approach have been published elsewhere [8]. In a new Policy Forum published in this issue of PLoS Medicine, Kavanagh [9] questions the external consistency of the GRADE framework by comparing the Surviving Sepsis Campaign (SSC) guideline recommendations developed in 2004 and updated in 2008. Moreover, Kavanagh expresses his concerns on the processes of the GRADE development and its formal validation. Had we likened the GRADE approach to an instrument or a health profile built on discrete logic to capture evidence, we would have concurred with some of Kavanagh’s criticism of GRADE. However, we see GRADE as a framework uncovering implicit subjectivity and invoking a systematic, explicit, judicious, and transparent approach to interpreting, as opposed to ‘‘capturing’’ evidence. It reveals how values are assigned to judgments, but what values are assigned it does not dictate simply because it cannot dictate. Below we first present our concern about one aspect of the GRADE framework and then our perspective on the various criticisms of it.

Impact of quality of evidence on the strength of recommendations: an empirical study

BackgroundEvidence is necessary but not sufficient for decision-making, such as making recommendations by clinical practice guideline panels. However, the fundamental premise of evidence-based medicine (EBM) rests on the assumed link between the quality of evidence and "truth" and/or correctness in making guideline recommendations. If this assumption is accurate, then the quality of evidence ought to play a key role in making guideline recommendations. Surprisingly, and despite the widespread penetration of EBM in health care, there has been no empirical research to date investigating the impact of quality of evidence on the strength of recommendations made by guidelines panels.MethodsThe American Association of Blood Banking (AABB) has recently convened a 12 member panel to develop clinical practice guidelines (CPG) for the use of fresh-frozen plasma (FFP) for 6 different clinical indications. The panel was instructed that 4 factors should play a role in making recommendation: quality of evidence, uncertainty about the balance between desirable (benefits) and undesirable effects (harms), uncertainty or variability in values and preferences, and uncertainty about whether the intervention represents a wise use of resources (costs). Each member of the panel was asked to make his/her final judgments on the strength of recommendation and the overall quality of the body of evidence. "Voting" was anonymous and was based on the use of GRADE (Grading quality of evidence and strength of recommendations) system, which clearly distinguishes between quality of evidence and strength of recommendations.ResultsDespite the fact that many factors play role in formulating CPG recommendations, we show that when the quality of evidence is higher, the probability of making a strong recommendation for or against an intervention dramatically increases. Probability of making strong recommendation was 62% when evidence is "moderate", while it was only 23% and 13% when evidence was "low" or "very low", respectively.ConclusionWe report the first empirical evaluation of the relationship between quality of evidence pertinent to a clinical question and strength of the corresponding guideline recommendations. Understanding the relationship between quality of evidence and probability of making (strong) recommendation has profound implications for the science of quality measurement in health care.

Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies

The GRADE approach to grading the quality of evidence and strength of recommendations provides a comprehensive and transparent approach for developing clinical recommendations about using diagnostic tests or diagnostic strategies. Although grading the quality of evidence and strength of recommendations about using tests shares the logic of grading recommendations for treatment, it presents unique challenges. Guideline panels and clinicians should be alert to these special challenges when using the evidence about the accuracy of tests as the basis for clinical decisions. In the GRADE system, valid diagnostic accuracy studies can provide high quality evidence of test accuracy. However, such studies often provide only low quality evidence for the development of recommendations about diagnostic testing, as test accuracy is a surrogate for patient-important outcomes at best. Inferring from data on accuracy that using a test improves outcomes that are important to patients requires availability of an effective treatment, improved patients' wellbeing through prognostic information, or - by excluding an ominous diagnosis - reduction of anxiety and the opportunity for earlier search for an alternative diagnosis for which beneficial treatment can be available. Assessing the directness of evidence supporting the use of a diagnostic test requires judgments about the relationship between test results and patient-important consequences. Well-designed and conducted studies of allergy tests in parallel with efforts to evaluate allergy treatments critically will encourage improved guideline development for allergic diseases.

Grading Quality of Evidence and Strength of Recommendations for Diagnostic Tests and Strategies

Faced with the plethora of new diagnostic and therapeutic interventions, busy physicians need clear guidance on the best approaches to follow for their patients. This need has led to such a proliferation of practice guidelines (PGs)1 that for diabetes mellitus alone, for example, more than 150 guidelines are available worldwide. In the “jungle” of PGs, many provide conflicting guidance, and the literature displays extensive variation in the approaches to formulating recommendations. Therefore, there is an international move toward standardizing guideline methodology so that recommendations are conceived in a systematic and transparent process and that the link between the evidence and the strength of recommendations is explicitly documented. This commentary provides a brief overview of the principles for assessing the strength of evidence and the challenges guideline developers face in formulating graded recommendations related to the use of laboratory tests. Guidelines aim to close the gap between research and practice and to provide rigorously developed, valid, and applicable recommendations for achieving the best possible outcomes. The formulation of evidence-based guidelines implies a process in which the body of evidence has been systematically explored, its quality critically evaluated, and the research findings synthesized and translated into recommendations for best practice. In PGs, quality of evidence indicates the degree of confidence that the evidence is adequate to support recommendations. Quality of evidence can be judged by considering the following aspects (1): 1. Study design usually defines the level of evidence. For example, questions on the efficacy of treatment are best answered by randomized controlled trials (RCTs), and questions about diagnostic accuracy are best addressed by properly designed prospective cohort studies. 2. Internal validity refers to a lack of design-related biases that could threaten the soundness of the study. In diagnostic accuracy studies, various forms of verification biases, spectrum bias, or review bias can lead …

Grading quality of evidence and strength of recommendations in clinical practice guidelines

The GRADE (Grades of Recommendation, Assessment, Development, and Evaluation) approach provides guidance to grading the quality of underlying evidence and the strength of recommendations in health care. The GRADE system's conceptual underpinnings allow for a detailed stepwise process that defines what role the quality of the available evidence plays in the development of health care recommendations. The merit of GRADE is not that it eliminates judgments or disagreements about evidence and recommendations, but rather that it makes them transparent. This first article in a three-part series describes the GRADE framework in relation to grading the quality of evidence about interventions based on examples from the field of allergy and asthma. In the GRADE system, the quality of evidence reflects the extent to which a guideline panel's confidence in an estimate of the effect is adequate to support a particular recommendation. The system classifies quality of evidence as high, moderate, low, or very low according to factors that include the study methodology, consistency and precision of the results, and directness of the evidence.