Grading Of Recommendations Assessment, Development And Evaluation Guidance Research Articles

A number of factors explain why WHO guideline developers make strong recommendations inconsistent with GRADE guidance

ObjectiveMany strong recommendations issued by the World Health Organization (WHO) are based on low- or very low-quality (low certainty) evidence (discordant recommendations). Many such discordant recommendations are inconsistent with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. We sought to understand why WHO makes discordant recommendations inconsistent with GRADE guidance. Study Design and SettingWe interviewed panel members involved in guidelines approved by WHO (2007–2012) that included discordant recommendations. Interviews, recorded and transcribed, focused on use of GRADE including the reasoning underlying, and factors contributing to, discordant recommendations. ResultsFour themes emerged: strengths of GRADE, challenges and barriers to GRADE, strategies to improve GRADE application, and explanations for discordant recommendations. Reasons for discordant recommendations included skepticism about the value of making conditional recommendations; political considerations; high certainty in benefits (sometimes warranted, sometimes not) despite assessing evidence as low certainty; and concerns that conditional recommendations will be ignored. ConclusionWHO panelists make discordant recommendations inconsistent with GRADE guidance for reasons that include limitations in their understanding of GRADE. Ensuring optimal application of GRADE at WHO and elsewhere likely requires selecting panelists who have a commitment to GRADE principles, additional training of panelists, and formal processes to maximize adherence to GRADE principles.

Users identified challenges in applying GRADE to complex interventions and suggested an extension to GRADE

ObjectivesTo explore user perspectives on applying the Grading of Recommendations Assessment, Development and Evaluation (GRADE) to systematic reviews of complex interventions. Study Design and SettingThirty-three authors of recent (2013 onward) systematic reviews were contacted regarding their perspectives on using GRADE from three Cochrane review groups: Cochrane Developmental, Psychosocial, and Learning Problems Group; Cochrane Public Health Group; and Cochrane Depression, Anxiety, and Neurosis Group. Framework Analysis was applied to the data to identify the challenges in applying GRADE and suggestions for its extension, that is, adaptation. These two themes were cross-compared between the groups of answers from “simple” vs. “complex” intervention review authors to identify the specific perspectives on using GRADE in reviews of complex interventions. ResultsSpecific challenges were identified in applying GRADE to reviews of complex interventions. These were related to the assessment of nonrandomized studies and performance bias in GRADE. Authors perceived these challenges to contribute to frequent downgrading of the “best evidence possible” for complex interventions. Meanwhile, GRADE was found to lack an analytic approach to enable adequate evidence synthesis and assessment of intervention implementation elements. ConclusionUsers suggest that the GRADE guidance be extended to address-specific considerations for complex interventions.

Senior GRADE methodologists encounter challenges as part of WHO guideline development panels: an inductive content analysis

BackgroundThe World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. ObjectiveTo gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations. Data sourcesSenior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published article on GRADE methodology. MethodsFive eligible methodologists participated in detailed semistructured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes. ResultsThree themes emerged from the interviews of five methodologists: (1) The perceived role of methodologists in the process, (2) Contributors to discordant recommendations, and (3) Strategies for improvement. Salient findings included (1) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; (2) both financial and nonfinancial conflicts of interest among panel members as an explanation for discordant recommendations; and (3) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels. ConclusionsThese findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO.

An algorithm was developed to assign GRADE levels of evidence to comparisons within systematic reviews

ObjectivesOne recommended use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is supporting quality assessment of evidence of comparisons included within a Cochrane overview of reviews. Within our overview, reviewers found that current GRADE guidance was insufficient to make reliable and consistent judgments. To support our ratings, we developed an algorithm to grade quality of evidence using concrete rules. MethodsUsing a pragmatic, exploratory approach, we explored the challenges of applying GRADE levels of evidence and developed an algorithm to applying GRADE levels of evidence in a consistent and transparent approach. Our methods involved application of algorithms and formulas to samples of reviews, expert panel discussion, and iterative refinement and revision. ResultsThe developed algorithm incorporated four key criteria: number of participants, risk of bias of trials, heterogeneity, and methodological quality of the review. A formula for applying GRADE level of evidence from the number of downgrades assigned by the algorithm was agreed. ConclusionOur algorithm which assigns GRADE levels of evidence using a set of concrete rules was successfully applied within our Cochrane overview. We propose that this methodological approach has implications for assessment of quality of evidence within future evidence syntheses.