Evidence Synthesis Approaches Research Articles

An international modified Delphi process supported updating the web-based "right review" tool

ObjectivesThe proliferation of evidence synthesis methods makes it challenging for reviewers to select the ‘‘right’’ method. This study aimed to update the Right Review tool (a web-based decision support tool that guides users through a series of questions for recommending evidence synthesis methods) and establish a common set of questions for the synthesis of both quantitative and qualitative studies (https://rightreview.knowledgetranslation.net/). Study Design and SettingA 2-round modified international electronic modified Delphi was conducted (2022) with researchers, health-care providers, patients, and policy makers. Panel members rated the importance/clarity of the Right Review tool's guiding questions, evidence synthesis type definitions and tool output. High agreement was defined as at least 70% agreement. Any items not reaching high agreement after round 2 were discussed by the international Project Steering Group. ResultsTwenty-four experts from 9 countries completed round 1, with 12 completing round 2. Of the 46 items presented in round 1, 21 reached high agreement. Twenty-seven items were presented in round 2, with 8 reaching high agreement. The Project Steering Group discussed items not reaching high agreement, including 8 guiding questions, 9 review definitions (predominantly related to qualitative synthesis), and 2 output items. Three items were removed entirely and the remaining 16 revised and edited and/or combined with existing items. The final tool comprises 42 items; 9 guiding questions, 25 evidence synthesis definitions and approaches, and 8 tool outputs. ConclusionThe freely accessible Right Review tool supports choosing an appropriate review method. The design and clarity of this tool was enhanced by harnessing the Delphi technique to shape ongoing development. The updated tool is expected to be available in Quarter 1, 2025.

Implementing GRADE in systematic reviews that adhere to JBI methodological conduct.

GRADE is a methodological approach used to establish certainty in a body of evidence and is now widely adopted among the evidence synthesis and guideline development community. JBI is an international evidence-based health care organization that provides guidance for a range of evidence synthesis approaches. The GRADE approach is currently endorsed for use in a subset of JBI systematic reviews; however, there is some uncertainty regarding when (and how) GRADE may be implemented in reviews that follow JBI methodology.

Seeing the forest (plot) for the trees-the importance of evidence synthesis in older adult care.

Systematically reviewing all the available evidence and then creating summary analyses of the pooled data is the foundation of evidence-based practice. Indeed, this evidence synthesis approach informs much of the care of older adults in hospital and community. It is perhaps no surprise that the journal Age and Ageing is a frequent platform for publishing research papers based on systematic review and synthesis. This research has evolved substantially from the early days of evidence-based medicine and the Cochrane Collaboration. The traditional approach would be a quantitative summary, calculated using pair-wise meta-analysis of randomised controlled trials of drug versus placebo, or a synthesis of observational studies to create summaries of prevalence, associations and outcomes. Methods have evolved and newer techniques such as scoping reviews, test accuracy meta-analysis and qualitative synthesis are all now available. The sophistication of these methods is driven in part by the increasingly complex decisions that need be made in contemporary older adult care. Age and Ageing continues to champion established and novel evidence synthesis approaches, and in the accompanying Collection exemplars of these differing methods are presented and described. Whilst there is marked heterogeneity in the techniques used, the consistent and defining feature of all these papers is the desire to comprehensively, and critically summarise the evidence in order to answer the most pertinent questions regarding older adult care.

Policy Studies Yearbook Annual Review 2022–23: Note from the editor and call for papers

<i>Policy Studies Yearbook Annual Review</i> 2022–23: Note from the editor and call for papers

Investigating different typologies for the synthesis of evidence: a scoping review protocol.

The objective of this scoping review is to identify evidence synthesis types and previously proposed classification systems, typologies, or taxonomies that have guided evidence synthesis. Evidence synthesis is a constantly evolving field. There is now a plethora of evidence synthesis approaches used across many different disciplines. Historically, there have been numerous attempts to organize the types and methods of evidence synthesis in the form of classification systems, typologies, or taxonomies. This scoping review will seek to identify all the available classification systems, typologies, or taxonomies; how they were developed; their characteristics; and the types of evidence syntheses included within them. This scoping review will include discussion papers, commentaries, books, editorials, manuals, handbooks, and guidance from major organizations that describe multiple approaches to evidence synthesis in any discipline. The Evidence Synthesis Taxonomy Initiative will support this scoping review. The search strategy will aim to locate both published and unpublished documents utilizing a three-step search strategy. An exploratory search of MEDLINE has identified keywords and MeSH terms. A second search of MEDLINE, Embase, CINAHL with Full Text, ERIC, Scopus, Compendex, and JSTOR will be conducted. The websites of relevant evidence synthesis organizations will be searched. Identified documents will be independently screened, selected, and extracted by two researchers, and the data will be presented in tables and summarized descriptively. Open Science Framework https://osf.io/qwc27.

Individual participant data sharing intentions and practices during the coronavirus disease-2019 pandemic: A rapid review

Abstract The coronavirus disease-2019 (COVID-19) pandemic has led to the irrational use of drugs in the absence of clinical management guidelines. Access to individual participant data (IPD) from clinical trials aids the evidence synthesis approaches. We undertook a rapid review to infer IPD sharing intentions based on data availability statements by the principal investigators (PIs) of drug and vaccine trials in the context of COVID-19. Searches were conducted on PubMed (NCBI). We considered randomized controlled trial (RCT) publications from January 1, 2020, to October 31, 2021. IPD sharing intentions were inferred based on the data availability statements in the full-text manuscript publications. We included 180 articles. Of these, 81.7% (147/180) of the publications have arrived from the findings of the RCTs alone, 12.8% (23/180) of the publications were protocol publications alone, and 5.6% (10/180) of the RCTs had both published protocol and publication from the trial findings. We have reported IPD sharing intentions separately in RCT protocol publications (n = 23 + 10) and publications from RCT findings (n = 147 + 10). Among RCT protocol publications, one-third (11/33) of the PIs intended to share IPD. In fact, over half of the PIs (52.2%, 82/157) in their published RCT findings intended to share IPD. However, information to share about IPD was missing for 57.6% (19/33) of RCT protocols and 38.2% (60/157) of published RCT findings. Stakeholders must work together to ensure that overarching factors, such as legislation that governs clinical trial practices, are streamlined to bolster IPD sharing mechanisms.

Beyond the Buzzword: A Framework for an Indigenous Relational Evaluation in Traditional Communities in Ghana

The notions of indigenous knowledge and cultural philosophies are becoming ubiquitous in many social inquiries, and evaluation is no exception. Nonetheless, the pursuit to embed relevant indigenous philosophies in contemporary evaluation has yet to succeed. In this article, we discuss indigenous relational philosophies, approaches, and practices as they relate to evaluation. Using qualitative research approaches, we interviewed 43 Indigenous development leaders and other local representatives in three local government areas in Ghana. Utilizing evidence synthesis approaches through a triangulation process, we conclude that indigenous knowledge and other cultural ethos were distinct in community-based development evaluation processes. There was an elusive intersection between indigenous and contemporary evaluation paradigms. Indigenous evaluation has principles such as community spirit, mutual trust, self-organization, relational patterns or networks, “ubuntu” ideals, consensus building, and collective action that can complement contemporary evaluation for the effective and efficient evaluation of community development programs and social policies. We identify key indigenous elements and other indigenous relational assessment patterns to aid in the design of an indigenously driven relational evaluation framework. The evaluative competencies embedded in indigenous philosophies are vast, thus, a call for future research is proposed.

Rapid evidence synthesis to enable innovation and adoption in health and social care

BackgroundThe rapid identification and adoption of effective innovations in healthcare is a known challenge. The strongest evidence base for innovations can be provided by evidence synthesis, but this is frequently a lengthy process and even rapid versions of this can be time-consuming and complex. In the UK, the Accelerated Access Review and Academic Health Science Network (AHSN) have provided the impetus to develop a consistently rapid process to support the identification and adoption of high-value innovations in the English NHS.MethodsThe Greater Manchester Applied Research Collaboration (ARC-GM) developed a framework for a rapid evidence synthesis (RES) approach, which is highly integrated within the innovation process of the Greater Manchester AHSN and the associated healthcare and research ecosystem. The RES uses evidence synthesis approaches and draws on the GRADE Evidence to Decision framework to provide rapid assessments of the existing evidence and its relevance to specific decision problems. We implemented this in a real-time context of decision-making around adoption of innovative health technologies.ResultsKey stakeholders in the Greater Manchester decision-making process for healthcare innovations have found that our approach is both timely and flexible; it is valued for its combination of rigour and speed.Our RES approach rapidly and systematically identifies, appraises and contextualises relevant evidence, which can then be transparently incorporated into decisions about the wider adoption of innovations. The RES also identifies limitations in existing evidence for innovations and this can inform subsequent evaluations. There is substantial interest from other ARCs and AHSNs in implementing a similar process. We are currently exploring methods to make completed RES publicly available. We are also exploring methods to evaluate the impact of using RES as more implementation decisions are made.ConclusionsThe RES framework we have implemented combines transparency and consistency with flexibility and rapidity. It therefore maximises utility in a real-time decision-making context for healthcare innovations.

JBI series paper 2: tailored evidence synthesis approaches are required to answer diverse questions: a pragmatic evidence synthesis toolkit from JBI

Evidence synthesis is critical in evidence-based healthcare and is a core program of JBI. JBI evidence synthesis is characterised by a pluralistic view of what constitutes evidence and is underpinned by a pragmatic ethos to facilitate the use of evidence to inform practice and policy. This second paper in this series provides a descriptive overview of the JBI evidence synthesis toolkit with reference to resources for 11 different types of reviews. Unique methodologies such as qualitative syntheses, mixed methods reviews, and scoping reviews are highlighted. Key features include standardised and collaborative processes for development of methodologies and a broad range of tailored resources to facilitate the conduct of a JBI evidence synthesis, including appraisal and data extraction tools, software to support the conduct of a systematic review and an intensive systematic review training program. JBI is one of the leading international protagonists for evidence synthesis, providing those who want to answer health-related questions with a toolkit of resources to synthesize the evidence.

9.F. Workshop: Rapid evidence synthesis to inform the national response to the COVID-19 pandemic in Ireland

Abstract The Health Information and Quality Authority (HIQA) is an independent statutory authority in Ireland. Since March 2020, HIQA has been conducting evidence syntheses to support decision-making by the Irish National Public Health Emergency Team (NPHET). This work has informed guidance developed by national and international agencies such as the European Centre for Disease Prevention and Control (ECDC), Newfoundland and Labrador Center for Applied Health Research, Sciensano (Belgium), Alberta Health Services and Sante Publique France. The HIQA COVID-19 Evidence Synthesis Team draws on a broad range of evidence with expert clinical and public health input from HIQA's COVID-19 Expert Advisory Group, and produces a range of outputs to best suit the evidence needs of decision-makers. Across the world, agencies have had to adapt rapidly to provide up-to-date COVID-19 research evidence to decision-makers. Similarly, evidence users, including the public, have had to assimilate information on COVID-19 generated by such agencies. Significant challenges exist given the speed at which the evidence base is evolving on COVID-19, its sometimes conflicting nature and often poor quality, and the subsequent influence of its dissemination on policy and on individual behaviour and risk perceptions. This workshop describes the work of HIQA's COVID-19 Evidence Synthesis Team in its role to support the national response to the COVID-19 pandemic. The aim of this workshop is to share learnings on the structure, methodological approaches, and impact of a COVID-19 Evidence Synthesis Team. The presenters who all work on, or are affiliated with HIQA's COVID-19 Evidence Synthesis Team, will discuss a range of topics including the establishment of the team, the conduct of rapid evidence syntheses, and the impact and media coverage of the outputs. The presenters will each present their topic in turn and will allow ample time for engagement, and will use e-voting systems, to enable shared learning. The objectives for this workshop are to: Describe the establishment and organisation of a COVID-19 Evidence Synthesis Team Provide case examples of completed rapid evidence syntheses Examine the media coverage of selected outputs Present the findings of a an in-action evaluation Share learnings on the organisation of evidence synthesis teams, the conduct of rapid evidence syntheses and the subsequent communication of findings. As the pandemic moves into a new phase it is important that agencies involved in conducting, and using the findings of, evidence syntheses reflect on processes and impact to-date. Lessons learned from the COVID-19 evidence synthesis experience can inform the development of future evidence synthesis approaches as well as support future pandemic preparedness. There is a need to remain flexible and to innovate, and this workshop presents an ideal opportunity for a European public health audience to share experiences and ideas. Key messages The challenges and opportunities associated with conducting rapid evidence synthesis during a pandemic will be explored. This workshop will provide a space to share learnings and ideas.

Identifying Suicide and Self-Harm Research Priorities in North West England.

Background: Understanding and effectively managing self-harm and suicide require collaborative research between stakeholders focused on shared priorities. Aims: We aimed to develop a consensus about suicide and self-harm research priorities in the North West of England using the Delphi method. Method: Items for the Delphi survey were generated through group discussions at a workshop with 88 stakeholders and subsequent thematic analysis of key themes. A total of 44 participants who were experts-by-experience, researchers, and clinicians based within health services including third-sector organizations completed the Delphi survey. Results: A three-round survey reached consensus on 55 research priority items identifying key priorities in each of the following groups: offenders, children and young people, self-harm in community settings, and crisis care in the community. Limitations: The pool of delegates at the workshop and subsequent self-selection into the Delphi may have introduced bias into the study. Conclusion: The current paper highlights specific actionable priorities were identified in four areas that can be used to inform research efforts and future policy and practice, based on shared areas of perceived importance and concern. Future work is needed to confirm the significance of these priority areas, including the use of evidence synthesis approaches to ascertain the extent to which these priorities have already been investigated and where gaps in understanding remain.

Harnessing Telemedicine for the Provision of Health Care: Bibliometric and Scientometric Analysis.

BackgroundIn recent decades, advances in information technology have given new momentum to telemedicine research. These advances in telemedicine range from individual to population levels, allowing the exchange of patient information for diagnosis and management of health problems, primary care prevention, and education of physicians via distance learning.ObjectiveThis scientometric investigation aims to examine collaborative research networks, dominant research themes and disciplines, and seminal research studies that have contributed most to the field of telemedicine. This information is vital for scientists, institutions, and policy stakeholders to evaluate research areas where more infrastructural or scholarly contributions are required.MethodsFor analyses, we used CiteSpace (version 4.0 R5; Drexel University), which is a Java-based software that allows scientometric analysis, especially visualization of collaborative networks and research themes in a specific field.ResultsWe found that scholarly activity has experienced a significant increase in the last decade. Most important works were conducted by institutions located in high-income countries. A discipline-specific shift from radiology to telestroke, teledermatology, telepsychiatry, and primary care was observed. The most important innovations that yielded a collaborative influence were reported in the following medical disciplines, in descending order: public environmental and occupational health, psychiatry, pediatrics, health policy and services, nursing, rehabilitation, radiology, pharmacology, surgery, respiratory medicine, neurosciences, obstetrics, and geriatrics.ConclusionsDespite a continuous rise in scholarly activity in telemedicine, we noticed several gaps in the literature. For instance, all the primary and secondary research central to telemedicine was conducted in the context of high-income countries, including the evidence synthesis approaches that pertained to implementation aspects of telemedicine. Furthermore, the research landscape and implementation of telemedicine infrastructure are expected to see exponential progress during and after the COVID-19 era.

Evidence synthesis methodology for questions relating to barriers and enablers in health care: a scoping review protocol.

The objective of the scoping review is to map the range of methodologies used to undertake evidence synthesis aimed at determining barriers and enablers in health care. The number of evidence synthesis studies focusing on identifying factors that support/facilitate (act as "enablers") or impede (act as "barriers") implementation of best practice in health care has significantly increased as researchers and clinicians strive to understand the evidence-practice gap. There is currently no consensus on the evidence synthesis approach for such topics, with the methods for analyzing or synthesizing results from barrier and enabler studies varying across existing evidence reviews. This scoping review will consider various evidence synthesis approaches that identify barriers and enablers in health care. The review will only consider evidence synthesis approaches that describe their methods (e.g. systematic reviews or scoping reviews rather than literature reviews). Barriers and enablers could be at different levels of the health system, including consumers of care, health practitioners, health organization, policy makers, and the community. The scoping review will focus on the methodological aspects or processes related to the review of primary studies on barriers and/or enablers. The proposed scoping review will be conducted in accordance with JBI methodology for scoping reviews. Studies will be searched in PubMed (NLM), Embase (Elsevier), CINAHL (EBSCOhost), PsycINFO (Ovid), Cochrane Database of Systematic Reviews, JBI Evidence Synthesis, and EPPI-Centre Systematic Reviews. Data extraction will be undertaken using a purpose-built data extraction form. A narrative summary will accompany the tabulated data/figures, with a view to proposing recommendations related to the methodology for undertaking evidence synthesis of barrier and enabler studies.

WITHDRAWN: Evaluating impact from research: A methodological framework

Abstract Background Interest in impact evaluation has grown rapidly as research funders increasingly demand evidence that their investments lead to public benefits. Aims This paper analyses literature to provide a new definition of research impact and impact evaluation, develops a typology of research impact evaluation designs, and proposes a methodological framework to guide evaluations of the significance and reach of impact that can be attributed to research. Method An adapted Grounded Theory Analysis of research impact evaluation frameworks drawn from cross-disciplinary peer-reviewed and grey literature. Results • Recognizing the subjective nature of impacts as they are perceived by different groups in different times, places and cultures, we define research impact evaluation as the process of assessing the significance and reach of both positive and negative effects of research. • Five types of impact evaluation design are identified encompassing a range of evaluation methods and approaches: i) experimental and statistical methods; ii) textual, oral and arts-based methods; iii) systems analysis methods; iv) indicator-based approaches; and v) evidence synthesis approaches. • Our guidance enables impact evaluation design to be tailored to the aims and context of the evaluation, for example choosing a design to establish a body of research as a necessary (e.g. a significant contributing factor among many) or sufficient (e.g. sole, direct) cause of impact, and choosing the most appropriate evaluation design for the type of impact being evaluated. Conclusion Using the proposed definitions, typology and methodological framework, researchers, funders and other stakeholders working across multiple disciplines can select a suitable evaluation design and methods to evidence the impact of research from any discipline.

Scoping Reviews, Systematic Reviews, and Meta-Analysis: Applications in Veterinary Medicine

Evidence-based decision making is a hallmark of effective veterinary clinical practice. Scoping reviews, systematic reviews, and meta-analyses all are methods intended to provide transparent and replicable ways of summarizing a body of research to address an important clinical or public health issue. As these methods increasingly are being used by researchers and read by practitioners, it is important to understand the distinction between these techniques and to understand what research questions they can, and cannot, address. This review provides an overview of scoping reviews, systematic reviews, and meta-analysis, including a discussion of the method and uses. A sample dataset and coding to conduct a simple meta-analysis in the statistical program R also are provided. Scoping reviews are a descriptive approach, designed to chart the literature around a particular topic. The approach involves an extensive literature search, following by a structured mapping, or charting, of the literature. The results of scoping reviews can help to inform future research by identifying gaps in the existing literature and also can be used to identify areas where there may be a sufficient depth of literature to warrant a systematic review. Systematic reviews are intended to address a specific question by identifying and summarizing all of the available research that has addressed the review question. Questions types that can be addressed by a systematic review include prevalence/incidence questions, and questions related to etiology, intervention efficacy, and diagnostic test accuracy. The systematic review process follows structured steps with multiple reviewers working in parallel to reduce the potential for bias. An extensive literature search is undertaken and, for each relevant study identified by the search, a formal extraction of data, including the effect size, and assessment of the risk of bias is performed. The results from multiple studies can be combined using meta-analysis. Meta-analysis provides a summary effect size, and allows heterogeneity of effect among studies to be quantified and explored. These evidence synthesis approaches can provide scientific input to evidence-based clinical decision-making for veterinarians and regulatory bodies, and also can be useful for identifying gaps in the literature to enhance the efficiency of future research in a topic area.

VP89 A Preliminary Equity Checklist To Support The HTA Process

IntroductionThere is increased recognition of the need to include equity considerations in HTA. Despite this, a recent World Health Organization report has found that this is seldom the case. We developed a preliminary version of an equity checklist in the hopes that tangible guidance will increase such analyses in the future and contribute to smart capability building.MethodsThe checklist is based on the Equity Framework for HTA developed by Culyer &amp; Bombard (2012). The elements presented in the framework were revised to follow the stepwise HTA process. A comprehensive literature search was used to update and complete the elements. The checklist was then piloted in an HTA in 2018 and subsequently further refined through a workshop during a national HTA conference in Canada.ResultsThese steps resulted in a 27-item checklist leading to consider different aspects of the three major phases in the HTA process. The scoping phase brings questions relative to defining and contextualizing equity, such as highlighting potential minority groups and including vulnerability factors in the logic model. The development phase leads methodological approaches facilitating the analysis of inequities as well as considering contextual realities leading to inequities. The last phase, drafting of recommendations, aims to be aware of the evidence synthesis approaches as well as the various aspects to ensure recommendations consider existing inequities and avoid contributing to their development.ConclusionsGiven the essence of HTA to protect health by ensuring optimal technologies and interventions are adopted to the benefit of all system users, the consideration of inequities should constitute an integral part of its process. The use of a pragmatic and simple checklist to aid the planning of an HTA could contribute to greater consideration of inequities in the future. A movement in this direction could also lead to greater methodological developments for health equity analysis in HTA.

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

BackgroundScoping reviews are a relatively new approach to evidence synthesis and currently there exists little guidance regarding the decision to choose between a systematic review or scoping review approach when synthesising evidence. The purpose of this article is to clearly describe the differences in indications between scoping reviews and systematic reviews and to provide guidance for when a scoping review is (and is not) appropriate.ResultsResearchers may conduct scoping reviews instead of systematic reviews where the purpose of the review is to identify knowledge gaps, scope a body of literature, clarify concepts or to investigate research conduct. While useful in their own right, scoping reviews may also be helpful precursors to systematic reviews and can be used to confirm the relevance of inclusion criteria and potential questions.ConclusionsScoping reviews are a useful tool in the ever increasing arsenal of evidence synthesis approaches. Although conducted for different purposes compared to systematic reviews, scoping reviews still require rigorous and transparent methods in their conduct to ensure that the results are trustworthy. Our hope is that with clear guidance available regarding whether to conduct a scoping review or a systematic review, there will be less scoping reviews being performed for inappropriate indications better served by a systematic review, and vice-versa.

Methods for Developing Evidence Reviews in Short Periods of Time: A Scoping Review.

IntroductionRapid reviews (RR), using abbreviated systematic review (SR) methods, are becoming more popular among decision-makers. This World Health Organization commissioned study sought to summarize RR methods, identify differences, and highlight potential biases between RR and SR.MethodsReview of RR methods (Key Question 1 [KQ1]), meta-epidemiologic studies comparing reliability/ validity of RR and SR methods (KQ2), and their potential associated biases (KQ3). We searched Medline, EMBASE, Cochrane Library, grey literature, and checked reference lists, used personal contacts, and crowdsourcing (e.g. email listservs). Selection and data extraction was conducted by one reviewer (KQ1) or two reviewers independently (KQ2-3).ResultsAcross all KQs, we identified 42,743 citations through the literature searches. KQ1: RR methods from 29 organizations were reviewed. There was no consensus on which aspects of the SR process to abbreviate. KQ2: Studies comparing the conclusions of RR and SR (n = 9) found them to be generally similar. Where major differences were identified, it was attributed to the inclusion of evidence from different sources (e.g. searching different databases or including different study designs). KQ3: Potential biases introduced into the review process were well-identified although not necessarily supported by empirical evidence, and focused mainly on selective outcome reporting and publication biases.ConclusionRR approaches are context and organization specific. Existing comparative evidence has found similar conclusions derived from RR and SR, but there is a lack of evidence comparing the potential of bias in both evidence synthesis approaches. Further research and decision aids are needed to help decision makers and reviewers balance the benefits of providing timely evidence with the potential for biased findings.

GetReal: from efficacy in clinical trials to relative effectiveness in the real world.

The GetReal consortium ("incorporating real-life data into drug development") addresses the efficacy-effectiveness gap that opens between the data from well-controlled randomized trials in selected patient groups submitted to regulators and the real-world evidence on effectiveness and safety of drugs required by decision makers. Workpackage 4 of GetReal develops evidence synthesis and modelling approaches to generate the real-world evidence. In this commentary, we discuss how questions change when moving from the well-controlled randomized trial setting to real-life medical practice, the evidence required to answer these questions, the populations to which estimates will be applicable to and the methods and data sources used to produce these estimates. We then introduce the methodological reviews written by GetReal authors and published in Research Synthesis Methods on network meta-analysis, individual patient data meta-analysis and mathematical modelling to predict drug effectiveness. The critical reviews of key methods are a good starting point for the ambitious programme of work GetReal has embarked on. The different strands of work under way in GetReal have great potential to contribute to making clinical trials research as relevant as it can be to patients, caregivers and policy makers. Copyright © 2016 John Wiley & Sons, Ltd.

Evaluation of Pharmacy and Therapeutic (P&amp;T) Committee member knowledge, attitudes and ability regarding the use of comparative effectiveness research (CER) in health care decision-making

Comparative effectiveness research (CER) is a constellation of research methods designed to improve health care decision making. Educational programs that improve health care decision makers' CER knowledge and awareness may ultimately lead to more cost-effective use of health care resources. This study was conducted to evaluate changes in CER knowledge, attitudes, and ability among Pharmacy and Therapeutics (P&T) Committee members and support staff after attending a tailored educational program. Physicians and pharmacists from two professional societies and the Indian Health Service who participated in the P&T process were invited via email to participate in this study. Participants completed a questionnaire, designed specifically for this study, prior to and following the 4-hour live, educational program on CER to determine the impact on their related knowledge, attitudes, and ability to use CER in decision-making. Rasch analysis was used to assess validity and reliability of subsections of the questionnaire and regression analysis was used to assess programmatic impact on CER knowledge, attitude, and ability. One hundred and forty of the 199 participants completed both the pre- and post-CER session questionnaires (response rate=70.4%). Most participants (>75%) correctly answered eight of the ten knowledge items after attending the educational session. More than 60% of the respondents had a positive attitude toward CER both before and after the program. Compared to baseline (pretest), participants reported significant improvements in their perceived ability to use CER after attending the session in these areas: using CER reviews, knowledge of CER methods, identifying problems with randomized controlled trials, identifying threats to validity, understanding of evidence synthesis approaches, and evaluating the quality of CER (all P values<0.001). The questionnaire demonstrated acceptable reliability and validity evidence (limited evidence of construct under-representation and construct irrelevant variance). The CER educational program was effective in increasing participants' CER knowledge and self-perceived ability to evaluate relevant evidence. Improving knowledge and awareness of CER and its applicability is a critical first step in improving its use in health care decision making.