Use Of Systematic Reviews Research Articles

The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review.

BackgroundSystematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which make them hard to use for decision-making. There are many organizations that develop and disseminate derivative products, such as evidence summaries, from systematic reviews for different populations or subsets of decision makers. This systematic review aimed to (1) assess the effectiveness of evidence summaries on policymakers’ use of the evidence and (2) identify the most effective summary components for increasing policymakers’ use of the evidence. We present an overview of the available evidence on systematic review derivative products.MethodsWe included studies of policymakers at all levels as well as health system managers. We included studies examining any type of “evidence summary,” “policy brief,” or other products derived from systematic reviews that presented evidence in a summarized form. The primary outcomes were the (1) use of systematic review summaries in decision-making (e.g., self-reported use of the evidence in policymaking and decision-making) and (2) policymakers’ understanding, knowledge, and/or beliefs (e.g., changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g., format) of the summaries.ResultsOur database search combined with our gray literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text, and we included six studies (reported in seven papers) as well as protocols from two ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (four studies), and perceived usefulness or usability (three studies). Summary of findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for summary of findings tables, certain elements, such as reporting study event rates and absolute differences, were preferred as well as avoiding the use of footnotes.ConclusionsEvidence summaries are likely easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.Trial registrationThe protocol was published in the journal Systematic Reviews (2015;4:122)Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-016-0530-3) contains supplementary material, which is available to authorized users.

Evidence and Health Policy: Using and Regulating Systematic Reviews.

Systematic reviews have, increasingly, informed policy for almost 3 decades. In many countries, systematic reviews have informed policy for public and population health, paying for health care, increasing the quality and efficiency of interventions, and improving the effectiveness of health sector professionals and the organizations in which they work. Systematic reviews also inform other policy areas: criminal justice, education, social welfare, and the regulation of toxins in the environment. Although the production and use of systematic reviews has steadily increased, many clinicians, public health officials, representatives of commercial organizations, and, consequently, policymakers who are responsive to them, have been reluctant to use these reviews to inform policy; others have actively opposed using them. Systematic reviews could inform policy more effectively with changes to current practices and the assumptions that sustain these practices-assumptions made by researchers and the organizations that employ them, by public and private funders of systematic reviews, and by organizations that finance, set priorities and standards for, and publish them.

Evaluation of a training program for medicines-oriented policymakers to use a database of systematic reviews.

BackgroundSuboptimal prescribing and medications use is a problem for health systems globally. Systematic reviews are a comprehensive resource that can help guide evidence-informed decision-making and implementation of interventions addressing such issues; however, a barrier to the use of systematic reviews is their inaccessibility (due to both dispersion across journals and inaccessibility of content). Publicly available databases, such as Rx for Change, provide quick access to summaries of appraised systematic reviews of professional and consumer-oriented interventions to improve prescribing behaviour and appropriate medication use, and may help maximise the use of evidence to inform decisions. The present study aims to evaluate a training program to improve attitudes towards, confidence in skills, intentions to use, and use of systematic review evidence contained within Rx for Change.MethodsGuided by the Knowledge to Action framework, a training program with content customised to local provider and consumer contexts was developed with knowledge user input. The training program consisted of a 6 minute information video, a 1 hour workshop with hands-on, interactive and didactic components, and two post-training reminders. Forty-nine people from five medicines-focused organisations in Canada and Australia attended one of six workshops. Participants were surveyed immediately pre and post and 3 months after training to evaluate their attitudes towards, confidence in skills, intentions to use, and use of Rx for Change, and attitudes towards and confidence in skills for using evidence for decision-making. Analyses for differences for each of the outcomes at three time points (pre, post and 3 months after training) was performed using a random effects model.ResultsImmediately post-training, there were higher respondent attitudes towards Rx for Change (mean increase = 0.54 out of 5, 95% CI, 0.18–0.83, P < 0.005); intention to use Rx for Change (0.53, 95% CI, 0.21–0.86, P < 0.005); confidence in skills for using Rx for Change (2.08, 95% CI, 1.74–2.42, P < 0.005); and confidence in skills for using evidence in policy decision-making (0.50, 95% CI, 0.22–0.77, P < .005) compared to pre-training. Confidence in skills for using both Rx for Change and evidence were maintained 3 months after training (both P < 0.005).ConclusionsParticipants of this training program reported sustained improvements in their confidence in skills for using evidence in policy decision-making. This may have important implications for uptake of systematic review evidence promoting improved prescribing and medication use.

Evidência científica para intervenções em crianças com síndrome de Down

A partir dos resultados de estudos, conduzidos no período 2009-2013, para identificar evidências científicas eficazes das intervenções fonoaudiológicas – em linguagem oral ou escrita – em crianças com síndrome de Down, apenas uma revisão sistemática da literatura (RSL) foi encontrada. Esses dados sugerem a necessidade de se conduzir pesquisas, para fomentar outras revisões sistemáticas, concebidas como investigações que reúnem, analisam e sintetizam estudos primários para identificar evidências científicas que contribuam para o êxito de intervenções em crianças com problemas fonológicos. Para tanto, foram rastreadas 554 RSL, e encontrada apenas uma que trata de intervenções relacionadas à consciência fonológica, reconhecidas como eficazes para melhoria do desempenho em leitura oral. Pesquisadores de RSL recomendam aperfeiçoar seu uso para a prática clínica e sugerem novos estudos que utilizem o delineamento de pesquisa experimental intrasujeito.

Empirical evaluation of the Q-Genie tool: a protocol for assessment of effectiveness

IntroductionMeta-analyses of genetic association studies are affected by biases and quality shortcomings of the individual studies. We previously developed and validated a risk of bias tool for use in systematic...

SYSTEMATIC REVIEW AND META-ANALYSIS: PITFALLS OF METHODS

Methods that allow to systematize the accumulated information on specifically formulated questions, analyze it and make reliable findings are of great interest and popularity with the emergence of a huge number of studies and publications in various fields of medicine, a significant increase in information, failure to identify weak effects in the individual studies. The use of systematic reviews and high-quality meta-analysis is an analytical basis of evidence-based medicine, it is a very valuable tool for decision-making at the level of practitioner, health care leaders and experts in the preparation of guidelines and legislation concerning the use of the most effective and safe treatments, when planning future research, in the development of rational health policy. Features of methods of systematic review and meta-analysis and their strengths and weaknesses are discussed in the article. The authors present the historical facts of occurrence of the Cochrane Collaboration, whose main task is to systematize and analyze accumulated information from a variety of medical research. The contribution of Russian researchers into this field of medical science is also assessed.

Dissemination of Evidence From Systematic Reviews Through Academic CME Providers: A Feasibility Study.

Although systematic reviews represent a source of best evidence to support clinical decision-making, reviews are underutilized by clinicians. Barriers include lack of awareness, familiarity, and access. Efforts to promote utilization have focused on reaching practicing clinicians, leaving unexplored the roles of continuing medical education (CME) directors and faculty in promoting systematic review use. This study explored the feasibility of working with CME directors and faculty for that purpose. A convenience sample of five academic CME directors and faculty agreed to participate in a feasibility study exploring use in CME courses of systematic reviews from the Agency for Healthcare Research and Quality (AHRQ-SRs). AHRQ-SR topics addressed the comparative effectiveness of health care options. Participants received access to AHRQ-SR reports, associated summary products, and instructional resources. The feasibility study used mixed methods to assess 1) implementation of courses incorporating SR evidence, 2) identification of facilitators and barriers to integration, and 3) acceptability to CME directors, faculty, and learners. Faculty implemented 14 CME courses of varying formats serving 1700 learners in urban, suburban, and rural settings. Facilitators included credibility, conciseness of messages, and availability of supporting materials; potential barriers included faculty unfamiliarity with SRs, challenges in maintaining review currency, and review scope. SR evidence and summary products proved acceptable to CME directors, course faculty, and learners by multiple measures. This study demonstrates the feasibility of approaches to use AHRQ-SRs in CME courses/programming. Further research is needed to demonstrate generalizability to other types of CME providers and other systemic reviews.

Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches.

BackgroundThere is an increasing interest worldwide to ensure evidence-informed health policymaking as a means to improve health systems performance. There is a need to engage policymakers in collaborative approaches to generate and use knowledge in real world settings. To address this gap, we implemented two interventions based on iterative exchanges between researchers and policymakers/implementers. This article aims to reflect on the implementation and impact of these multi-site evidence-to-policy approaches implemented in low-resource settings.MethodsThe first approach was implemented in Mexico and Nicaragua and focused on implementation research facilitated by communities of practice (CoP) among maternal health stakeholders. We conducted a process evaluation of the CoPs and assessed the professionals’ abilities to acquire, analyse, adapt and apply research. The second approach, called the Policy BUilding Demand for evidence in Decision making through Interaction and Enhancing Skills (Policy BUDDIES), was implemented in South Africa and Cameroon. The intervention put forth a ‘buddying’ process to enhance demand and use of systematic reviews by sub-national policymakers. The Policy BUDDIES initiative was assessed using a mixed-methods realist evaluation design.ResultsIn Mexico, the implementation research supported by CoPs triggered monitoring by local health organizations of the quality of maternal healthcare programs. Health programme personnel involved in CoPs in Mexico and Nicaragua reported improved capacities to identify and use evidence in solving implementation problems. In South Africa, Policy BUDDIES informed a policy framework for medication adherence for chronic diseases, including both HIV and non-communicable diseases. Policymakers engaged in the buddying process reported an enhanced recognition of the value of research, and greater demand for policy-relevant knowledge.ConclusionsThe collaborative evidence-to-policy approaches underline the importance of iterations and continuity in the engagement of researchers and policymakers/programme managers, in order to account for swift evolutions in health policy planning and implementation. In developing and supporting evidence-to-policy interventions, due consideration should be given to fit-for-purpose approaches, as different needs in policymaking cycles require adapted processes and knowledge. Greater consideration should be provided to approaches embedding the use of research in real-world policymaking, better suited to the complex adaptive nature of health systems.Electronic supplementary materialThe online version of this article (doi:10.1186/s12961-016-0089-0) contains supplementary material, which is available to authorized users.

Systematic Review of Methods in Low-Consensus Fields: Supporting Commensuration through `Construct-Centered Methods Aggregation’ in the Case of Climate Change Vulnerability Research

There is increasing interest in using systematic review to synthesize evidence on the social and environmental effects of and adaptations to climate change. Use of systematic review for evidence in this field is complicated by the heterogeneity of methods used and by uneven reporting. In order to facilitate synthesis of results and design of subsequent research a method, construct-centered methods aggregation, was designed to 1) provide a transparent, valid and reliable description of research methods, 2) support comparability of primary studies and 3) contribute to a shared empirical basis for improving research practice. Rather than taking research reports at face value, research designs are reviewed through inductive analysis. This involves bottom-up identification of constructs, definitions and operationalizations; assessment of concepts’ commensurability through comparison of definitions; identification of theoretical frameworks through patterns of construct use; and integration of transparently reported and valid operationalizations into ideal-type research frameworks. Through the integration of reliable bottom-up inductive coding from operationalizations and top-down coding driven from stated theory with expert interpretation, construct-centered methods aggregation enabled both resolution of heterogeneity within identically named constructs and merging of differently labeled but identical constructs. These two processes allowed transparent, rigorous and contextually sensitive synthesis of the research presented in an uneven set of reports undertaken in a heterogenous field. If adopted more broadly, construct-centered methods aggregation may contribute to the emergence of a valid, empirically-grounded description of methods used in primary research. These descriptions may function as a set of expectations that improves the transparency of reporting and as an evolving comprehensive framework that supports both interpretation of existing and design of future research.

The Role of Systematic Review in the Practice of Toxicology and Risk Assessment An Appreciation for the Primary Tool in Evidence-Based Practice

Use of systematic reviews (SRs) as a tool to facilitate evidence-based toxicology (EBT) assessments is increasing, though the field has yet to develop an appreciation of the rigor required to appropriately utilize this tool. Toxicologists should recognize the weight of the term, understanding that a systematic review involves far more than conducting elements of a review systematically. Key aspects that appear to be currently underappreciated include development and publication of a protocol, the level of documentation involved in the conduct of a SR, and the overall level of effort required to maintain standards of SR. As many regulatory agencies and health organizations integrate systematic review into their procedures, it is clear that there is a need to develop best-science practices in EBT, as the methods developed for evidence-based medicine (EBM) do not always provide the best platform for evaluation of toxicological data. Such efforts are particularly needed for evidence integration, methods which allow for integration of multiple types of data, as well as application of the SR in both qualitative and quantitative hazard or risk assessments. Nonetheless, use of systematic review is advancing the field of toxicology, providing objectivity and transparency in evidence-based assessments.

Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review.

BackgroundWe completed a scoping review on the barriers and facilitators to use of systematic reviews by health care managers and policy makers, including consideration of format and content, to develop recommendations for systematic review authors and to inform research efforts to develop and test formats for systematic reviews that may optimise their uptake.MethodsWe used the Arksey and O’Malley approach for our scoping review. Electronic databases (e.g., MEDLINE, EMBASE, PsycInfo) were searched from inception until September 2014. Any study that identified barriers or facilitators (including format and content features) to uptake of systematic reviews by health care managers and policy makers/analysts was eligible for inclusion. Two reviewers independently screened the literature results and abstracted data from the relevant studies. The identified barriers and facilitators were charted using a barriers and facilitators taxonomy for implementing clinical practice guidelines by clinicians.ResultsWe identified useful information for authors of systematic reviews to inform their preparation of reviews including providing one-page summaries with key messages, tailored to the relevant audience. Moreover, partnerships between researchers and policy makers/managers to facilitate the conduct and use of systematic reviews should be considered to enhance relevance of reviews and thereby influence uptake.ConclusionsSystematic review authors can consider our results when publishing their systematic reviews. These strategies should be rigorously evaluated to determine impact on use of reviews in decision-making.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-016-0370-1) contains supplementary material, which is available to authorized users.

Evidence-based causation in toxicology: A 10-year retrospective.

We introduced Evidence-based Toxicology (EBT) in 2005 to address the disparities that exist between the various Weight-of-Evidence (WOE) methods typically applied in the regulatory hazard decision-making arena and urged toxicologists to adopt the evidence-based guidelines long-utilized in medicine (i.e., Evidence-Based Medicine or EBM). This review of the activities leading to the adoption of evidence-based methods and EBT during the last decade demonstrates how fundamental concepts that form EBT, such as the use of systematic reviews to capture and consider all available information, are improving toxicological evaluations performed by various groups and agencies. We reiterate how the EBT framework, a process that provides a method for performing human chemical causation analyses in an objective, transparent and reproducible manner, differs significantly from past and current regulatory WOE approaches. We also discuss why the uncertainties associated with regulatory WOE schemes lead to a definition of the term "risk" that contains unquantifiable uncertainties not present in this term as it is used in epidemiology and medicine. We believe this distinctly different meaning of "risk" should be clearly conveyed to those not familiar with this difference (e.g., the lay public), when theoretical/nomologic risks associated with chemical-induced toxicities are presented outside of regulatory and related scientific parlance.

Probiotics in 2015: Their Scope and Use.

The field of probiotics continues to evolve and progress. This paper reviews several situations within the probiotic field that are of current interest, including review of the scope of the proper use of the term "probiotic," use of systematic reviews and meta-analyses for probiotics, regulatory challenges to doing human research on probiotics in the United States, medical recommendations for probiotic use, and safety assurance for probiotics used for vulnerable populations. The greatest need in the probiotic field remains well-conducted and well-reported human trials, to better define the functionality of probiotics for different indications and populations.

Do evidence summaries increase policy-makers' use of evidence from systematic reviews: A systematic review protocol.

BackgroundSystematic reviews are important for decision-makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. There are many organizations that develop and disseminate derivative products, such as evidence summaries, from systematic reviews for different populations or subsets of decision-makers. This systematic review will assess the effectiveness of systematic review summaries on increasing policymakers’ use of systematic review evidence and to identify the components or features of these summaries that are most effective.Methods/designWe will include studies of policy-makers at all levels as well as health-system managers. We will include studies examining any type of “evidence summary,” “policy brief,” or other products derived from systematic reviews that present evidence in a summarized form. The primary outcomes are the following: (1) use of systematic review summaries decision-making (e.g., self-reported use of the evidence in policy-making, decision-making) and (2) policy-maker understanding, knowledge, and/or beliefs (e.g., changes in knowledge scores about the topic included in the summary). We will conduct a systematic review of randomized controlled trials (RCTs), non-randomized controlled trials (NRCTs), controlled before-after studies (CBA), and interrupted time series (ITS) studies.DiscussionThe results of this review will inform the development of future systematic review summaries to ensure that systematic review evidence is accessible to and used by policy-makers making health-related decisions.Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-015-0116-1) contains supplementary material, which is available to authorized users.

Evidence synthesis in international development: a critique of systematic reviews and a pragmatist alternative

Systematic reviews are an instrument of Evidence-Based Policy designed to produce comprehensive, unbiased, transparent and clear assessments of interventions’ effectiveness. From their origins in medical fields, systematic reviews have recently been promoted as offering important advances in a range of applied social science fields, including international development. Drawing on a case study of a systematic review of the effectiveness of community mobilisation as an intervention to tackle HIV/AIDS, this article problematises the use of systematic reviews to summarise complex and context-specific bodies of evidence. Social development interventions, such as ‘community mobilisation’ often take different forms in different interventions; are made successful by their situation in particular contexts, rather than being successful or unsuccessful universally; and have a rhetorical value that leads to the over-application of positively valued terms (e.g. ‘community mobilisation’), invalidating the keyword search process of a systematic review. The article suggests that the policy interest in definitive summary statements of ‘the evidence’ is at odds with academic assessments that evidence takes multiple, contradictory and complex forms, and with practitioner experience of the variability of practice in context. A pragmatist philosophy of evidence is explored as an alternative. Taking this approach implies expanding the definition of forms of research considered to be ‘useful evidence’ for evidence-based policy-making; decentralising decisions about ‘what works’ to allow for the use of local practical wisdom; and prioritising the establishment of good processes for the critical use of evidence, rather than producing context-insensitive summaries of ‘the evidence’.

Assessing the usefulness of systematic reviews for policymakers in public health: A case study of overweight and obesity prevention interventions

Objective: Systematic reviews (SRs) should include policy-relevant information in order to more readily inform policy and practice. We investigated whether SRs of overweight and obesity prevention interventions are framed in such a way that maximises their usefulness for policymakers. Method: We conducted a systematic review of SRs of overweight and obesity prevention interventions published in 4 databases any time up to December 2014. We analysed the SRs for their usefulness to policymakers, using a coding frame developed based on literature around what policymakers want and need from systematic reviews. Systematic reviews were assessed for a) policy links and framing; b) quality assessment and conflict of interest statements: and c) discussion of policy implications. Results: Of the 153 SRs that met the inclusion criteria, very few (7%) had authors from policy-based organisations, 48% had funding from such organisations, and almost a third (31%) framed their introduction or aims around policy. Most (69%) discussed issues affecting generalisability of the SR findings but only a quarter (24%) discussed cost or cost-effectiveness of the intervention under investigation. Less than a third (29%) of SRs discussed the policy implications of their findings. SRs that were policy-framed were significantly more likely to discuss costs (PR=1.8, 95%CI 1.0–3.0) and policy implications (PR=2.5, 95%CI 1.5–4.0). Conclusion: SRs should discuss the policy and practice implications of their findings to maximise the influence of SRs on policy making. It is recommended that SR guidelines are updated to include generalisability and discussion of policy and practice implications as a requirement.

Use of Systematic Review and Meta-Analysis in Environmental Health Epidemiology: a Systematic Review and Comparison with Guidelines.

Systematic review (SR) and meta-analysis (MA) have potential to contribute substantially to environmental health (EH) risk assessment and policy-making, provided study questions are clear and methods sound. We undertook a systematic review of the published epidemiological literature for studies using both SR and MA examining associations between chronic low-dose chemical exposures and adverse health outcomes in general populations and compared actual methods and reporting with a checklist based on available published guidelines. We identified 48 EH SRMAs meeting these criteria. Associations were mainly positive and statistically significant, often involving large populations. A majority of studies followed most general SRMA guidance, although we identified weaknesses in problem formulation, study search, selection and data extraction, and integrating policy implications. Fewer studies followed EH-specific SRMA recommendations, particularly regarding exposure heterogeneity and other risks of bias. Development and adoption of EH-specific SRMA guidelines would contribute to strengthening these tools for public health decision-making.Electronic supplementary materialThe online version of this article (doi:10.1007/s40572-015-0062-z) contains supplementary material, which is available to authorized users.

The use of systematic reviews in the design of randomized controlled trials of acupuncture: a systematic review

The use of systematic reviews in the design of randomized controlled trials of acupuncture: a systematic review

Characteristics and practices of National Immunisation Technical Advisory Groups in Europe and potential for collaboration, April 2014.

In many countries, national vaccination recommendations are developed by independent expert committees, so-called national immunisation technical advisory groups (NITAG). Since the evaluation of vaccines is complex and resource-demanding, collaboration between NITAGs that evaluate the same vaccines could be beneficial. We conducted a cross-sectional survey among 30 European countries in February 2014, to explore basic characteristics and current practices of European NITAGs and identify potential modes and barriers for collaboration. Of 28 responding countries, 26 reported to have a NITAG or an equivalent expert group. Of these, 20 apply a systematic approach in the vaccine decision-making process, e.g. by considering criteria such as country-specific disease epidemiology, vaccine efficacy/effectiveness/safety, health economics, programme implementation/logistics or country-specific values/preferences. However, applied frameworks and extent of evidence review differ widely. The use of systematic reviews is required for 15 of 26 NITAGs, while results from transmission modelling and health economic evaluations are routinely considered by 18 and 20 of 26 NITAGs, respectively. Twenty-five countries saw potential for NITAG-collaboration, but most often named structural concerns, e.g. different NITAG structures or countries’ healthcare systems. Our survey gathered information that can serve as an inventory on European NITAGs, allowing further exploration of options and structures for NITAG collaboration.

Evidence-based pathology: systematic literature reviews as the basis for guidelines and best practices.

Evidence-based medicine has been proposed as a new paradigm for the identification and evaluation of medical information. Best available evidence or data are identified and used as the basis for the diagnosis and treatment of individual patients. Evidence-based pathology has adapted basic evidence-based medicine concepts to the specific needs of pathology and laboratory medicine. To briefly review the history and basic concepts of evidence-based medicine and evidence-based pathology, describe how to perform and interpret systematic reviews, and discuss how to integrate best evidence into guidelines. PubMed (National Library of Medicine, Washington, DC) and Web of Science (Thompson Reuters, New York, New York) were used. Evidence-based pathology provides methodology to evaluate the quality of information published in pathology journals and apply it to the diagnosis of tissue samples and other tests from individual patients. Information is gathered through the use of systematic reviews, using a method that is less biased and more comprehensive than ad hoc literature searches. Published data are classified into evidence levels to provide readers with a quick impression about the quality and probable clinical validity of available information. Best available evidence is combined with personal experience for the formulation of evidence-based, rather than opinion-based, guidelines that address specific practice needs.