Evidence For Development Of Guidelines Research Articles

Using real-world data alongside evidence in guideline development

Using real-world data alongside evidence in guideline development

A quality appraisal using the AGREE II instrument of endotracheal tube suction guidelines in neonatal intensive care units

BackgroundEndotracheal tube (ETT) suction is among the most common procedures performed in neonatal intensive care units (NICUs). Although necessary, it is associated with significant risks. To mitigate these risks, clinical practice guidelines are developed to provide evidence-based recommendations. ObjectiveThe aim of the study was to appraise the quality of neonatal ETT suction guidelines from all NICUs in Australia and New Zealand. MethodsAll level III NICUs in Australia and New Zealand were invited to participate. Three researchers graded the methodological quality of the received guidelines using the AGREE II instrument. Item and domain scores were calculated by scaling as a percentage of the total possible score out of 100%. A threshold score of <50% is considered to be of limited potential use. ResultsTwenty-three (79.31%) clinical practice guidelines were received from 29 invited facilities. The scaled results of the AGREE II domains were as follows: Scope and Purpose, mean = 73%, 95% confidence interval (CI) = 63–83%; Stakeholder Involvement, mean = 23%, 95% CI = 15–31%; Rigour of Development, mean = 17%, 95% CI = 12–21%; Clarity of Presentation, mean = 63%, 95% CI = 56–70%; Applicability, mean = 5%, 95% CI = 20–30%; and Editorial Independence, mean = 50%, 95% CI = 50–50%. Overall assessment indicated low methodological quality (31%; 22–39%), with only five clinical practice guidelines scoring >50%, suggesting that they could be recommended for use with modifications. The remaining 18 could not be recommended for use. ConclusionsNeonatal ETT suction guidelines are of a low methodological quality. All guidelines poorly incorporated latest evidence in guideline development. This appraisal highlights the need to improve the quality of neonatal ETT suction guidelines to promote optimal patient care.

Use of evidence for clinical practice guideline development.

Clinical practice guidelines provide a framework against which quality of care is measured. Recommendations contained within guidelines are used for decision-making not only within the clinical domain but also other related issues within the health systems. As such the use of research evidence for formulating recommendations contained in a guideline is a global standard to ensure guideline quality. The paper briefly reviews how the need for and use of evidence in guideline development shaped up historically and then provides an overview of the four typologies of guideline development mechanisms at the country level.

Guidelines for guideline developers: a systematic review of grading systems for medical tests

BackgroundA variety of systems have been developed to grade evidence and develop recommendations based on the available evidence. However, development of guidelines for medical tests is especially challenging given the typical indirectness of the evidence; direct evidence of the effects of testing on patient important outcomes is usually absent. We compared grading systems for medical tests on how they use evidence in guideline development.MethodsWe used a systematic strategy to look for grading systems specific to medical tests in PubMed, professional guideline websites, via personal correspondence, and handsearching back references of key articles. Using the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument as a starting point, we defined two sets of characteristics to describe these systems: methodological and process ones. Methodological characteristics are features relating to how evidence is gathered, appraised, and used in recommendations. Process characteristics are those relating to the guideline development process. Data were extracted in duplicate and differences resolved through discussion.ResultsTwelve grading systems could be included. All varied in the degree to which methodological and process characteristics were addressed. Having a clinical scenario, identifying the care pathway and/or developing an analytical framework, having explicit criteria for appraising and linking indirect evidence, and having explicit methodologies for translating evidence into recommendations were least frequently addressed. Five systems at most addressed these, to varying degrees of explicitness and completeness. Process wise, features most frequently addressed included involvement of relevant professional groups (8/12), external peer review of completed guidelines (9/12), and recommendations on methods for dissemination (8/12). Characteristics least often addressed were whether the system was piloted (3/12) and funder information (3/12).ConclusionsFive systems for grading evidence about medical tests in guideline development addressed to differing degrees of explicitness the need for and appraisal of different bodies of evidence, the linking of such evidence, and its translation into recommendations. At present, no one system addressed the full complexity of gathering, assessing and linking different bodies of evidence.

HIV and pregnancy

In resource rich settings transmission of HIV from mother to child during pregnancy and post partum has been significantly reduced by access to interventions such as maternal and neonatal antiretroviral therapy, avoidance of breast feeding and consideration to caesarean section. Accumulating observational and randomised controlled studies provide the evidence for development of guidelines for the clinical management of these women. However, despite referencing the same studies, differences exist between recommendations originating from the United States versus the United Kingdom. The particular areas of controversy include use of efavirenz, dose adjustment of antiretrovirals during pregnancy, mode of delivery according to maternal viral load, duration of neonatal zidovudine, use of PJP prophylaxis and number of antiretrovirals to prescribe in a neonate considered high risk of acquiring HIV infection. This article summarises these differences and suggests ways of approaching and adapting these conflicting recommendations to the local setting.

Prospective systematic review registration: perspective from the Guidelines International Network (G-I-N).

Clinical practice and public health guidelines are important tools for translating research findings into practice with the aim of assisting health practitioners as well as patients and consumers in health behavior and healthcare decision-making. Numerous programs for guideline development exist around the world, with growing international collaboration to improve their quality. One of the key features in developing trustworthy guidelines is that recommendations should be based on high-quality systematic reviews of the best available evidence. The review process used by guideline developers to identify and grade relevant evidence for developing recommendations should be systematic, transparent and unbiased. In this paper, we provide an overview of current international developments in the field of practice guidelines and methods to develop guidelines, with a specific focus on the role of systematic reviews. The Guidelines International Network (G-I-N) aims to stimulate collaboration between guideline developers and systematic reviewers to optimize the use of available evidence in guideline development and to increase efficiency in the guideline development process. Considering the significant benefit of systematic reviews for the guideline community, the G-I-N Board of Trustees supports the international prospective register of systematic reviews (PROSPERO) initiative. G-I-N also recently launched a Data Extraction Resource (GINDER) to present and share data extracted from individual studies in a standardized template. PROSPERO and GINDER are complementary tools to enhance collaboration between guideline developers and systematic reviewers to allow for alignment of activities and a reduction in duplication of effort.

Incorporation of economic evidence in the Dutch guideline ‘cardiovascular risk management’

The consideration of economic evidence in guideline development may be particularly important in health care management when different (drug) therapies show similar efficacy on clinical endpoints, such as in cardiovascular diseases. This article investigates to what extent the Dutch guideline 'cardiovascular risk management' (2006) considers cost-effectiveness and budget impact according to the most recent economic evidence. We carried out a systematic review of economic evaluations on cholesterol-lowering drugs and antihypertensives followed by an assessment of guideline recommendations. The guideline does not consider the most recent economic evidence but does consider cost-effectiveness based on economic evaluations performed in conjunction with clinical trials. Their conclusions are largely in agreement with the most recent economic evidence. An innovative aspect in the guideline is the application of a budget impact analysis to take accessibility and affordability constraints into account when considering cost-effectiveness. Based on the most recent economic evidence, the guideline could be improved by more firmly formulating recommendations in favour of cost-effective drug therapies (simvastatin, pravastatin and low-dose diuretics) to stimulate compliance to the guideline in clinical practice.

Methodology for clinical practice guidelines for the management of arteriovenous access

The Society for Vascular Surgery considers the placement and maintenance of arteriovenous hemodialysis access to be an important component of any vascular surgery practice. Therefore, the Society has long been involved in setting the standards for the management of arteriovenous access. Formulating clinical recommendations in this area is the latest effort by the Society to improve the management of arteriovenous access on a national level. To provide an unbiased study of the evidence and to help in formulating the recommendations, the Society used the Knowledge and Encounter Research (KER) Unit of the Mayo Clinic College of Medicine, Rochester, Minn, to review the available evidence and advise a multidisciplinary group of access surgeons and nephrologists in formulating the clinical recommendations. To review the evidence, randomized and observational study designs were both considered. Whenever possible, systematic reviews and meta-analyses of the literature were used because, compared with individual studies, they generate more precise estimates of treatment effects and their results are applicable to a wider range of patients. On behalf of the Society, the group issued its recommendations following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) format; this format disentangles the strength of recommendations from the quality of the evidence and encourages statements about the underlying values and preferences relevant to the particular recommendation. The recommendations are classified as strong (denoted by the phrase "we recommend") or weak (denoted by the phrase "we suggest"); and the quality of evidence is classified as high, moderate, low, or very low. These recommendations are not meant to supersede clinical judgment; rather, they should be used as a guide for the practicing surgeon and nephrologist as the decision is being made for the placement and subsequent procedures and management of arteriovenous hemodialysis access are being considered.

Improving the use of research evidence in guideline development: 11. Incorporating considerations of cost-effectiveness, affordability and resource implications

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 11th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on incorporating considerations of cost-effectiveness, affordability and resource implications in guidelines and recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWhen is it important to incorporate cost-effectiveness, resource implications and affordability considerations in WHO guidelines (which topics)?• For cost-effectiveness:The need for cost/effectiveness information should be dictated by the specific question, of which several may be addressed in a single guideline. It is proposed that the indications for undertaking a cost-effectiveness analysis (CEA) could be a starting point for determining which recommendation(s) in the guideline would benefit from such analysis.• For resource implications/affordability:The resource implications of each individual recommendation need to be considered when implementation issues are being discussed.How can cost-effectiveness, resource implications and affordability be explicitly taken into account in WHO guidelines?• For cost-effectiveness:∘ If data are available, the ideal time to consider cost-effectiveness is during the evidence gathering and synthesizing stage. However, because of the inconsistent availability of CEAs and the procedural difficulty associated with adjusting results from different CEAs to make them comparable, it is also possible for cost-effectiveness to be considered during the stage of developing recommendations.∘ Depending on the quantity and quality and relevance of the data available, such data can be considered in a qualitative way or in a quantitative way, ranging from a listing of the costs to a modelling exercise. At the very least, a qualitative approach like a commentary outlining the economic issues that need to be considered is necessary. If a quantitative approach is to be used, the full model should be transparent and comprehensive.• For resource implications/affordability:∘ Resource implications, including health system changes, for each recommendation in a WHO guideline should be explored. At the minimum, a qualitative description that can serve as a gross indicator of the amount of resources needed, relative to current practice, should be provided.How does one provide guidance in contextualizing guideline recommendations at the country level based on considerations of cost-effectiveness, resource implications and affordability?• All models should be made available and ideally are designed to allow for analysts to make changes in key parameters and reapply results in their own country.• In the global guidelines, scenarios and extensive sensitivity/uncertainty analysis can be applied.Resource implications for WHO• From the above, it is clear that guidelines development groups will need a health economist. There is need to ensure that this is included in the budget for guidelines and that there is in-house support for this as well.

Improving the use of research evidence in guideline development: 5. Group processes

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the fifth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectiveIn this review we address approaches to facilitate sound processes within groups that develop recommendations for health care.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key question and answerWhat should WHO do to ensure appropriate group processes?Various strategies can be adopted to ensure that the group processes in play when panels are developing recommendations are inclusive, so that all voices can be heard and all arguments given fair weight, including• the use of formal consensus development methods, such at the Nominal Group Technique or the Delphi method• the selection of a group leader who is qualified and responsible for facilitating an appropriate group process.

Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the thirteenth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on applicability, transferability, and adaptation of guidelines.MethodsWe searched five databases for existing systematic reviews and relevant primary methodological research. We reviewed the titles of all citations and retrieved abstracts and full text articles if the citations appeared relevant to the topic. We checked the reference lists of articles relevant to the questions and used snowballing as a technique to obtain additional information. We used the definition "coming from, concerning or belonging to at least two or all nations" for the term international. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe did not identify systematic reviews addressing the key questions. We found individual studies and projects published in the peer reviewed literature and on the Internet.Should WHO develop international recommendations?• Resources for developing high quality recommendations are limited. Internationally developed recommendations can facilitate access to and pooling of resources, reduce unnecessary duplication, and involve international scientists.• Priority should be given to international health problems and problems that are important in low and middle-income countries, where these advantages are likely to be greatest.• Factors that influence the transferability of recommendations across different settings should be considered systematically and flagged, including modifying factors, important variation in needs, values, costs and the availability of resources.What should be done centrally and locally?• The preparation of systematic reviews and evidence profiles should be coordinated centrally, in collaboration with organizations that produce systematic reviews. Centrally developed evidence profiles should be adaptable to specific local circumstances.• Consideration should be given to models that involve central coordination with work being undertaken by centres located throughout the world.• While needs, availability of resources, costs, the presence of modifying factors and values need to be assessed locally, support for undertaking these assessments may be needed to make guidelines applicable.• WHO should provide local support for adapting and implementing recommendations by developing tools, building capacity, learning from international experience, and through international networks that support evidence-informed health policies, such as the Evidence-informed Policy Network (EVIPNet).How should recommendations be adapted?• WHO should provide detailed guidance for adaptation of international recommendations.• Local adaptation processes should be systematic and transparent, they should involve stakeholders, and they should report the key factors that influence decisions, including those flagged in international guidelines, and the reasons for any modifications that are made.

Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the ninth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on grading evidence and recommendations in guidelines.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct a full systematic review ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersShould WHO grade the quality of evidence and the strength of recommendations?• Users of recommendations need to know how much confidence they can place in the underlying evidence and the recommendations. The degree of confidence depends on a number of factors and requires complex judgments. These judgments should be made explicitly in WHO recommendations. A systematic and explicit approach to making judgments about the quality of evidence and the strength of recommendations can help to prevent errors, facilitate critical appraisal of these judgments, and can help to improve communication of this information.What criteria should be used to grade evidence and recommendations?• Both the quality of evidence and the strength of recommendations should be graded. The criteria used to grade the strength of recommendations should include the quality of the underlying evidence, but should not be limited to that.• The approach to grading should be one that has wide international support and is suitable for a wide range of different types of recommendations. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, which is currently suggested in the Guidelines for WHO Guidelines, is being used by an increasing number of other organizations internationally. It should be used more consistently by WHO. Further developments of this approach should ensure its wide applicability.Should WHO use the same grading system for all of its recommendations?• Although there are arguments for and against using the same grading system across a wide range of different types of recommendations, WHO should use a uniform grading system to prevent confusion for developers and users of recommendations.

Improving the use of research evidence in guideline development: 16. Evaluation

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the last of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on evaluating guidelines and recommendations, including their quality, whether they are likely to be up-to-date, and their implementation. We also considered the role of guideline developers in undertaking evaluations that are needed to inform recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersOur answers to these questions were informed by a review of instruments for evaluating guidelines, several studies of the need for updating guidelines, discussions of the pros and cons of different research designs for evaluating the implementation of guidelines, and consideration of the use of uncertainties identified in systematic reviews to set research priorities.How should the quality of guidelines or recommendations be appraised?• WHO should put into place processes to ensure that both internal and external review of guidelines is undertaken routinely.• A checklist, such as the AGREE instrument, should be used.• The checklist should be adapted and tested to ensure that it is suitable to the broad range of recommendations that WHO produces, including public health and health policy recommendations, and that it includes questions about equity and other items that are particularly important for WHO guidelines.When should guidelines or recommendations be updated?• Processes should be put into place to ensure that guidelines are monitored routinely to determine if they are in need of updating.• People who are familiar with the topic, such as Cochrane review groups, should do focused, routine searches for new research that would require revision of the guideline.• Periodic review of guidelines by experts not involved in developing the guidelines should also be considered.• Consideration should be given to establishing guideline panels that are ongoing, to facilitate routine updating, with members serving fixed periods with a rotating membership.How should the impact of guidelines or recommendations be evaluated?• WHO headquarters and regional offices should support member states and those responsible for policy decisions and implementation to evaluate the impact of their decisions and actions by providing advice regarding impact assessment, practical support and coordination of efforts.• Before-after evaluations should be used cautiously and when there are important uncertainties regarding the effects of a policy or its implementation, randomised evaluations should be used when possible.What responsibility should WHO take for ensuring that important uncertainties are addressed by future research when the evidence needed to inform recommendations is lacking?• Guideline panels should routinely identify important uncertainties and research priorities. This source of potential priorities for research should be used systematically to inform priority-setting processes for global research.

Improving the use of research evidence in guideline development: 4. Managing conflicts of interests.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the fourth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on conflicts of interest to answer the following questions:1. What is the best way to obtain complete and accurate disclosures on financial ties and other competing interests?2. How to determine when a disclosed financial tie or other competing interest constitutes a conflict of interest?3. When a conflict of interest is identified, how should the conflict be managed?4. How could conflict of interest policies be enforced?MethodsWe searched PubMed, the Cochrane Methodology Register and selectively searched for the published policies of several organizations, We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWhat is the best way to obtain complete and accurate disclosures on financial ties and other competing interests?• Although there is little empirical evidence to guide the development of disclosure forms, minimal or open-ended formats are likely to be uninformative. We recommend the development of specific, detailed, structured forms that solicit as much information as possible about the nature and extent of the competing interests.How to determine when a disclosed financial tie or other competing interest constitutes a conflict of interest?• There is no empirical evidence to suggest that explicit criteria are preferable to ad hoc committee decisions when deciding if a disclosed financial tie is a conflict of interest. However, explicit criteria may make decision-making easier.When a conflict of interest is identified, how should the conflict be managed?• Descriptive studies suggest that appropriate management strategies are best determined on a case-by-case basis. Thus, WHO should use a wide range of management strategies to address disclosed conflicts of interest, with public disclosure of conflicts associated with each meeting as a minimum and recusal of conflicted individuals as the other extreme.How could conflict of interest policies be enforced?• Although there are no empirical studies of the enforcement of conflict if interest policies, descriptive studies of other organizations and institutions suggest that WHO convene a standing committee to review all financial disclosure statements prior to the commencement of committee meetings/hearings and to make management recommendations when necessary. A standard policy requiring all financial ties to be made public (i.e., recorded into the meeting minutes) should reduce the number of problematic cases. In instances where the conflicts seem intractable, a recommendation of recusal may be necessary to protect the greater interests of WHO and its constituents.

Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 10th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on integrating values and consumers in guideline development.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We reviewed the titles of all citations and retrieved abstracts and full text articles if the citations appeared relevant to the topic. We checked the reference lists of articles relevant to the questions and used snowballing as a technique to obtain additional information. We did not conduct a full systematic review ourselves. Our conclusions based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe did not find a systematic review of methods for integrating values in guidelines, but we found several systematic reviews that dealt with related topics.Whose values should WHO use when making recommendations?• Values, the relative importance or worth of a state or consequences of a decision (outcomes relating to benefits, harms, burden and costs), play a role in every recommendation. Ethical considerations, concepts that determine what is right, also play a role.• The values used in making recommendations should reflect those of the people affected. Judgements should be explicit and should be informed by input from those affected (including citizens, patients, clinicians and policy makers).• When differences in values may lead to different decisions or there is uncertainty about values, this should also be explicit. If differences in values are likely to affect a decision, such that people in different setting would likely make different choices about interventions or actions based on differences in their values, global recommendations should be explicit in terms of which values were applied and allow for adaptation after incorporating local values.How should WHO ensure that appropriate values are integrated in recommendations?• All WHO guideline groups should uniformly apply explicit, transparent and clearly described methods for integrating values.• WHO should consider involving relevant stakeholders if this is feasible and efficient.• WHO should develop a checklist for guidelines panels to help them to ensure that ethical considerations relevant to recommendations are addressed explicitly and transparently.How should users and consumers be involved in generating recommendations?• Including consumers in groups that are making global recommendations presents major challenges with respect to the impossibility of including a representative spectrum of consumers from a variety of cultures and settings. Nonetheless, consideration should be given to including consumers in groups who are able to challenge assumptions that are made about the values used for making recommendations, rather than represent the values of consumers around the world.• WHO should establish a network to facilitate involvement of users.• Draft recommendations should be reviewed by consumers, who should be asked explicitly to consider the values that were used.How should values be presented in recommendations?• Recommendations should include a description of how decisions were made about the relative importance of the consequences (benefits, harms and costs) of a decision.• Values that influence recommendations should be reported along with the research evidence underlying recommendations.• When differences in values would lead to different decisions or there is important uncertainty about values that are critical to a decision, this should be flagged and reflected in the strength of the recommendation.• Adaptable guideline templates that allow for integration of different values should be developed and used when differences in values are likely to be critical to a decision.

Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the eighth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on the synthesis and presentation of research evidence, focusing on four key questions.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe found two reviews of instruments for critically appraising systematic reviews, several studies of the importance of using extensive searches for reviews and determining when it is important to update reviews, and consensus statements about the reporting of reviews that informed our answers to the following questions.How should existing systematic reviews be critically appraised?• Because preparing systematic reviews can take over a year and require capacity and resources, existing reviews should be used when possible and updated, if needed.• Standard criteria, such as A MeaSurement Tool to Assess Reviews (AMSTAR), should be used to critically appraise existing systematic reviews, together with an assessment of the relevance of the review to the questions being asked.When and how should WHO undertake or commission new reviews?• Consideration should be given to undertaking or commissioning a new review whenever a relevant, up-to-date review of good quality is not available.• When time or resources are limited it may be necessary to undertake rapid assessments. The methods that are used to do these assessments should be reported, including important limitations and uncertainties and explicit consideration of the need and urgency of undertaking a full systematic review.• Because WHO has limited capacity for undertaking systematic reviews, reviews will often need to be commissioned when a new review is needed. Consideration should be given to establishing collaborating centres to undertake or support this work, similar to what some national organisations have done.How should the findings of systematic reviews be summarised and presented to committees responsible for making recommendations?• Concise summaries (evidence tables) of the best available evidence for each important outcome, including benefits, harms and costs, should be presented to the groups responsible for making recommendations. These should include an assessment of the quality of the evidence and a summary of the findings for each outcome.• The full systematic reviews, on which the summaries are based, should also be available to both those making recommendations and users of the recommendations.What additional information is needed to inform recommendations and how should this information be synthesised with information about effects and presented to committees?• Additional information that is needed to inform recommendations includes factors that might modify the expected effects, need (prevalence, baseline risk or status), values (the relative importance of key outcomes), costs and the availability of resources.• Any assumptions that are made about values or other factors that may vary from setting to setting should be made explicit.• For global guidelines that are intended to inform decisions in different settings, consideration should be given to using a template to assist the synthesis of information specific to a setting with the global evidence of the effects of the relevant interventions.

Improving the use of research evidence in guideline development: 12. Incorporating considerations of equity.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 12th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on incorporating considerations of equity in guidelines and recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe found few directly relevant empirical methodological studies. These answers are based largely on logical arguments.When and how should inequities be addressed in systematic reviews that are used as background documents for recommendations?• The following question should routinely be considered: Are there plausible reasons for anticipating differential relative effects across disadvantaged and advantaged populations?• If there are plausible reasons for anticipating differential effects, additional evidence should be included in a review to inform judgments about the likelihood of differential effects.What questions about equity should routinely be addressed by those making recommendations on behalf of WHO?• The following additional questions should routinely be considered:• How likely is it that the results of available research are applicable to disadvantaged populations and settings?• How likely are differences in baseline risk that would result in differential absolute effects across disadvantaged and advantaged populations?• How likely is it that there are important differences in trade-offs between the expected benefits and harms across disadvantaged and advantaged populations?• Are there different implications for disadvantaged and advantaged populations, or implications for addressing inequities?What context specific information is needed to inform adaptation and decision making in a specific setting with regard to impacts on equity?• Those making recommendations on behalf of WHO should routinely consider and offer advice about the importance of the following types of context specific data that might be needed to inform adaptation and decision making in a specific setting:• Effect modifiers for disadvantaged populations and for the likelihood of differential effects• Baseline risk in relationship to social and economic status• Utilization and access to care in relationship to social and economic status• Costs in relationship to social and economic status• Ethics and laws that may impact on strategies for addressing inequities• Availability of resources to address inequitiesWhat implementation strategies are likely be needed to ensure that recommendations are implemented equitably?• Organisational changes are likely to be important to address inequities. While it may only be possible to consider these in relationship to specific settings, consideration should be given to how best to provide support for identifying and addressing needs for organisational changes. In countries with pervasive inequities institutional, cultural and political changes may first be needed.• Appropriate indicators of social and economic status should be used to monitor the effects of implementing recommendations on disadvantaged populations and on changes in social and economic status.

Improving the use of research evidence in guideline development: 14. Reporting guidelines

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 14th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on reporting guidelines and recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersThere is little empirical evidence that addresses these questions. Our answers are based on logical arguments and standards put forward by other groups.What standard types of recommendations or reports should WHO use?• WHO should develop standard formats for reporting recommendations to facilitate recognition and use by decision makers for whom the recommendations are intended, and to ensure that all the information needed to judge the quality of a guideline, determine its applicability and, if needed, adapt it, is reported.• WHO should develop standard formats for full systematically developed guidelines that are sponsored by WHO, rapid assessments, and guidelines that are endorsed by WHO.• All three formats should include the same information as full guidelines, indicating explicitly what the group preparing the guideline did not do, as well as the methods that were used.• These formats should be used across clinical, public health and health systems recommendations.How should recommendations be formulated and reported?• Reports should be structured, using headings that correspond to those suggested by the Conference on Guideline Standardization or similar headings.• The quality of evidence and strength of recommendations should be reported explicitly using a standard approach.• The way in which recommendations are formulated should be adapted to the specific characteristics of a specific guideline.• Urgent attention should be given to developing a template that provides decision makers with the relevant global evidence that is needed to inform a decision and offers practical methods for incorporating the context specific evidence and judgements that are needed.

Improving the use of research evidence in guideline development: 6. Determining which outcomes are important.

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the sixth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on determining which outcomes are important for the development of guidelines.MethodsWe searched five databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct a complete systematic review ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWe did not find a systematic review that addresses any of the following key questions and we found limited relevant research evidence.What methods should WHO use to identify important outcomes?• Methods of outcome identification should be transparent and explicit.• The consultation process should start with identification of all relevant outcomes associated with an intervention.• Those affected, including consumers, should be involved in the selection of outcomes.• A question driven approach (what is important?) is preferable to a data driven approach (what data are at hand?) to identify important outcomes.What type of outcomes should WHO consider and how should cultural diversity be taken account of in the selection of outcomes?• Desirable (benefits, less burden and savings) and undesirable effects should be considered in all guidelines.• Undesirable effects include harms (including the possibility of unanticipated adverse effects), greater burden (e.g. having to go to the doctor) and costs (including opportunity costs).• Important outcomes (e.g. mortality, morbidity, quality of life) should be preferred over surrogate, indirect outcomes (e.g. cholesterol levels, lung function) that may or may not correlate with patient important outcomes.• Ethical considerations should be part of the evaluation of important outcomes (e.g. impacts on autonomy).• If the importance of outcomes is likely to vary across cultures, stakeholders from diverse cultures should be consulted and involved in the selection of outcomes.How should the importance of outcomes be ranked?• Outcomes should be ranked by relative importance, separated into benefits and downsides.• Information from research on values and preferences should inform the ranking of outcomes whenever possible.• If the importance of outcomes is likely to vary across cultures, ranking of outcomes should be done in specific settings.• If evidence is lacking for an important outcome, this should be acknowledged, rather than ignoring the outcome.

Improving the use of research evidence in guideline development: 7. Deciding what evidence to include

BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the seventh of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on what constitutes "evidence" in guidelines and recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key question and answersWe found several systematic reviews that compared the findings of observational studies with randomised trials, a systematic review of methods for evaluating bias in non-randomised trials and several descriptive studies of methods used in systematic reviews of population interventions and harmful effects.What types of evidence should be used to address different types of questions?• The most important type of evidence for informing global recommendations is evidence of the effects of the options (interventions or actions) that are considered in a recommendation. This evidence is essential, but not sufficient for making recommendations about what to do. Other types of required evidence are largely context specific.• The study designs to be included in a review should be dictated by the interventions and outcomes being considered. A decision about how broad a range of study designs to consider should be made in relationship to the characteristics of the interventions being considered, what evidence is available, and the time and resources available.• There is uncertainty regarding what study designs to include for some specific types of questions, particularly for questions regarding population interventions, harmful effects and interventions where there is only limited human evidence.• Decisions about the range of study designs to include should be made explicitly.• Great caution should be taken to avoid confusing a lack of evidence with evidence of no effect, and to acknowledge uncertainty.• Expert opinion is not a type of study design and should not be used as evidence. The evidence (experience or observations) that is the basis of expert opinions should be identified and appraised in a systematic and transparent way.