Comparative Effectiveness Reviews Research Articles

The American Psychiatric Association Practice Guideline for the Treatment of Patients With Eating Disorders.

The American Psychiatric Association Practice Guideline for the Treatment of Patients With Eating Disorders.

Eating disorders are associated with adverse obstetric and perinatal outcomes: a systematic review.

Objective:To systematically review the literature focusing on obstetric and perinatal outcomes in women with previous or current eating disorders (EDs) and on the consequences of maternal EDs for the offspring.Methods:The study was performed following the systematic review and meta-analysis (PRISMA) statement. PubMed, SciELO, and Cochrane databases were searched for non-interventional studies published in English or Portuguese from January 1980 to December 2020. Risk of bias was assessed using the Methods guide for effectiveness and comparative effectiveness reviews (American Agency for Healthcare Research and Quality).Results:The search yielded 441 records, and 30 articles were included. The psychiatric outcome associated with EDs in women was mainly perinatal depression. The most prevalent obstetric outcomes observed in women with EDs were vomiting, hyperemesis, bleeding, and anemia. Most studies found maternal anorexia nervosa and bulimia nervosa to be associated with low birth weight and slow fetal growth. Women with binge EDs delivered children with increased birth weight. Of the 30 studies included, methodological quality was good in seven, fair in eight, and poor in 15 studies.Conclusion:A considerable body of evidence was reviewed to assess obstetric and perinatal outcomes in EDs. Acute and lifetime EDs, especially if severe, correlated with poor perinatal, obstetric, and neonatal outcomes. Obstetricians and general practitioners should be vigilant and screen for EDs during pregnancy.

The American Psychiatric Association Practice Guideline for the Treatment of Patients With Schizophrenia.

(Reprinted with permission from The American Journal of Psychiatry 2020; 177:868-872).

The American Psychiatric Association Practice Guideline for the Treatment of Patients With Schizophrenia

The American Psychiatric Association Practice Guideline for the Treatment of Patients With Schizophrenia

Addressing heterogenous outcomes in uterine fibroid research: a call to action

Uterine fibroid tumors are the most common benign pelvic tumors in women, with complications that include heavy menstrual bleeding, pelvic pain, reproductive complications, and bulk-related symptoms. Although the majority of uterine fibroid tumors are asymptomatic, those women who experience symptoms can experience substantial burdens on quality of life and daily functioning. Comparative effectiveness reviews of available medical, surgical, and radiologic treatments have found that a lack of high-quality data to inform treatment decisions is, in part, due to the use of heterogeneous outcomes and instruments in clinical studies. With multiple new interventions emerging, this call-to-action encourages the development and use of a core outcome set that will capture the most relevant, patient-important outcomes in late-phase and after-marketing therapeutic trials for uterine fibroid tumors. The core outcome set should be developed by a diverse, multistakeholder group comprised of key healthcare decision-makers. Development and uptake of a core outcome set ensures that a consistent, collaboratively vetted set of outcomes will be accessible across different studies and promotes transparency for innovators who seek to anticipate the evidence needs of patients, providers, payers, regulators, and other stakeholders.

PDG91 OUTLYING STUDIES CAN CHANGE RANKING OF TREATMENTS AND COMPARATIVE EFFICACY ESTIMATES IN NETWORK META-ANALYSIS: FROM A COMPARATIVE STUDY OF ANTIHYPERTENSIVE DRUGS

Network meta-analysis is an evidence synthesis method by using direct and indirect evidence for all the treatment comparisons of interest. In many network meta-analyses, some studies have markedly different characteristics from the others. The inclusion of these “outlying” studies might lead biases and possibly produce misleading conclusions. The purpose of this study is to illustrate the influence of these outlying studies from an empirical evidence, a network meta-analysis of antihypertensive drugs (Sciarretta et al., 2011; Arch Intern Med 171: 384-394). We conducted network meta-analysis for 26 clinical trials (N=223,313) that compared α-blockers (AB), ACE inhibitors (ACEI), angiotensin II receptor blockers (ARB), β-blockers (BB), calcium channel blockers (CCB), conventional treatment (CT), diuretics, and placebo. The outcome was the incidence of heart failure. We assessed inconsistencies on the network of evidence using the Dias’s side-splitting method. We conducted sensitivity analyses that excluded candidates of outlying studies, which had markedly different characteristics. Through the side-splitting method, the direct and indirect evidence for the comparison of ARB vs. CT was significantly inconsistent (direct OR (odds-ratio): 1.49, indirect OR: 0.91, P = 0.027). After excluding three trials that involved ARB vs. CT (VALUE, E-COST, Jikei Heart Study), the treatment ranking has changed significantly. In the original analysis, it was diuretic-ACEI-ARB-CT-CCB-BB-placebo-AB, while in sensitivity analysis it was diuretic-CT-ACEI-CCB-ARB-BB-placebo-AB. The efficacy estimate of ARB got smaller, and that of ACEI got larger in the sensitivity analysis. In addition, CCB vs. placebo was significantly different (OR 0.79, 95%CI 0.64, 0.99; P=0.038) only in sensitivity analysis. Outlying studies possibly have influences to the overall conclusions of network meta-analyses, thus they should be carefully evaluated in practices. In addition, extreme profiles might be caused by certain biases such as industrial sponsorship bias. Sensitivity analyses are important to circumvent misleading conclusions for these comparative effectiveness reviews.

2050

OBJECTIVES/SPECIFIC AIMS: To improve the quality of evidence synthesis projects, including systematic reviews and other comparative effectiveness reviews, at the University of Utah. METHODS/STUDY POPULATION: Systematic reviews and other types of evidence syntheses are best when collaborative teams with expertise in multiple disciplines participate, including content experts, librarians and information specialists, systematic review methodologists, and statisticians. The Center for Clinical & Translational Science (CCTS), due to its interdisciplinary nature, connectivity to clinical experts, and existing Cores of methodologists, presented an opportune location for a Systematic Review Core. We designed the Systematic Review Core to focus on 2 primary aspects of evidence synthesis support: overall systematic review methodology guidance and in-depth information retrieval planning and execution. After establishing a conceptual partnership, a new position, Evidence Retrieval and Synthesis Librarian, was created to build capacity within the Core. RESULTS/ANTICIPATED RESULTS: Close connections with the CCTS’s Population Health Research Foundation have led to better interdisciplinary coverage of systematic reviews and other evidence syntheses produced by the University of Utah. We are able to partner with statisticians and clinical experts from formulating the question to completing the final manuscript. Hourly rates charged through a cost recovery model have enabled us to grow our staff able to work on the Core, as well as offset costs for major databases and resources these bibliographic data-heavy research methods require. After 1 year of existence, the Core is already at maximum capacity, with no sign of slowing. Projects have ranged from brief consultations to highly intense interactions for the duration of the research spectrum. We have also been added as key personnel to grants with systematic review components. DISCUSSION/SIGNIFICANCE OF IMPACT: Systematic reviews and other evidence syntheses are a labor-intense, interdisciplinary team effort that fit well within the scope of CTSA’s. They are a key component of the translation of science to practice, and can be used at all stages of the translational science spectrum. Quality of systematic reviews remains poor, particularly surrounding protocol development, sensitive search strategy design and reporting, and overall reporting. Librarians and information specialist involvement has been shown to positively correlate to the search strategy design and reporting aspects of systematic reviews, and librarians and information specialists increasingly act as systematic review methodologists. By including librarians and information specialists as part of the CTSA’s official Core structure, these systematic review methodologists are able to connect with statisticians, other methodologists, and clinical experts in a nexus of interdisciplinarity. At the University of Utah, the visibility and structure provided by the CCTS helps the Systematic Review Core with promotion, creating connections and opportunities for collaboration across the campus. This partnership has already led to increased uptake in services, and over time, we believe it will increase the quality of the science produced. CTSA’s have a natural partner with their health science library colleagues in translational science, as shown by this model.

The American Psychiatric Association Practice Guideline on the Use of Antipsychotics to Treat Agitation or Psychosis in Patients With Dementia.

(Reprinted with permission from American Journal of Psychiatry 2016; 173:543-546).

Distinguishing malignant and benign renal masses with composite models and nomograms: A systematic review and meta-analysis of clinically localized renal masses suspicious for malignancy.

Solid renal masses and cystic lesions with solid components are suspicious for renal cell carcinoma. Without an effective screening test, composite models and nomograms rely on patient and tumor characteristics to stratify the risk of benign disease versus malignant disease. To guide decisions about the use of renal mass sampling or excision, a systematic review and meta-analysis of the ability of composite models to predict the likelihood of malignancy on the basis of preoperative clinical variables was performed. MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials were searched from January 1, 1997, through May 1, 2015, according to the Preferred Reporting Items for Systematic Review and Meta-Analyses statement. Composite models necessarily included imaging results and at least 1 element from the following to be compared with surgical pathology: demographic characteristics, clinical characteristics, and blood or urine tests. Two independent reviewers screened citations and extracted data. Quality Assessment Tool for Diagnostic Accuracy Studies 2 was used to assess the risk of bias. The strength of evidence was graded with the scheme recommended by Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Twenty studies (12,149 patients) were included in this review. The only significant predictors of malignancy in the composite models were tumor size (effect size, 1.33-fold increased risk per centimeter; 95% confidence interval [CI], 1.22-1.43) and male sex (effect size, 2.71; 95% CI, 2.39-3.02). The results were inconclusive or not significant for tumor characteristics, age, body mass index, and incidental presentation. In conclusion, composite models currently have a limited ability to distinguish malignant renal masses from benign renal masses, with increased tumor size and male sex associated with malignancy. Cancer 2016;122:3267-3276. © 2016 American Cancer Society.

Change score or follow-up score? Choice of mean difference estimates could impact meta-analysis conclusions

ObjectivesIn randomized controlled clinical trials, continuous outcomes are typically measured at both baseline and follow-up, and mean difference could be estimated using the change scores from baseline or the follow-up scores. This study assesses the impact of using change score vs. follow-up score on the conclusions of meta-analyses. Study Design and SettingA total of 63 meta-analyses from six comparative effectiveness reviews were included. The combined mean difference was estimated using a random-effects model, and we also evaluated whether the impact qualitatively varied by alternative random-effects estimates. ResultsBased on the Dersimonian–Laird (DL) method, using the change vs. the follow-up score led to five meta-analyses (7.9%) showing discrepancy in conclusions. Based on the profile likelihood (PL) method, nine (14.3%) showed discrepancy in conclusions. Using change score was more likely to show a significant difference in effects between interventions (DL method: 4 of 5; PL method: 7 of 9). A significant difference in baseline scores did not necessarily lead to discrepancies in conclusions. ConclusionsUsing the change vs. the follow-up score could lead to important discrepancies in conclusions. Sensitivity analyses should be conducted to check the robustness of results to the choice of mean difference estimates.

The American Psychiatric Association Practice Guidelines for the Psychiatric Evaluation of Adults.

These Practice Guidelines for the Psychiatric Evaluation of Adults mark a transition in the American Psychiatric Association’s Practice Guidelines. Since the publication of the 2011 Institute of Medicine report Clinical Practice Guidelines We Can Trust, there has been an increasing focus on using clearly defined, transparent processes for rating the quality of evidence and the strength of the overall body of evidence in systematic reviews of the scientific literature. These guidelines were developed using a process intended to be consistent with the recommendations of the Institute of Medicine (2011), the Principles for theDevelopment of Specialty Society Clinical Guidelines of the Council of Medical Specialty Societies (2012), and the requirements of the Agency for Healthcare Research andQuality (AHRQ) for inclusion of a guideline in the National Guideline Clearinghouse. Parameters used for the guidelines’ systematic review are included with the full text of the guidelines; the development process is fully described in a document available on the APA website: http:// www.psychiatry.org/File%20Library/Practice/APA-GuidelineDevelopment-Process–updated-2011-.pdf. To supplement the expertise of members of the guideline work group, we used a “snowball” survey methodology to identify experts on psychiatric evaluation and solicit their input on aspects of the psychiatric evaluation that they saw as likely to improve specific patient outcomes (Yager 2014). Results of this expert survey are included with the full text of the practice guideline.

Effectiveness of culturally adapted Strengthening Families Programme 6-11 years among Portuguese families

Purpose – The purpose of this paper is to compare the outcomes from the Portuguese Strengthening Families Programme (SFP) with those from other countries to see if they are equally effective despite the new context. SFP was selected for cultural adaptation because comparative effectiveness reviews find that SFP is the most effective parenting and family intervention (Foxcroft et al., 2003, 2012). Standardised cultural adaptations of SFP have resulted in successful outcomes in 35 countries. Design/methodology/approach – The outcomes for the SFP six to 11 years Portuguese families (n=41) were compared to the SFP six to 11 years international norms (n=1,600) using a quasi-experimental, non-equivalent control two group pre- and post-test design. A 2×2 ANOVA generated the outcome tables including p-values and Cohen’s d effect sizes. Standardised test scales were used and measured 21 parenting, family and child risk and protective factors. Findings – Statistically significant positive results (p < 0.05) were found for 16 or 76.2 per cent of the 21 outcomes measured for Portuguese families. The Portuguese effect sizes were similar to the SFP international norms for improvements in the five parenting scales (d=0.61 vs 0.65), five family scales (d=0.68 vs 0.70) and seven children’s scales (d=0.48 vs 0.48) despite these norms having larger effect sizes than the USA norms. Hence, the cultural adaptation did not diminish the outcomes and SFP Portuguese families can benefit substantially from SFP participation. Originality/value – A Portuguese culturally adapted version of SFP had never been developed or evaluated; hence, this paper reports original findings.

The value of including observational studies in systematic reviews was unclear: a descriptive study

ObjectivesTo evaluate (1) how often observational studies are included in comparative effectiveness reviews (CERs); (2) the rationale for including observational studies; (3) how data from observational studies are appraised, analyzed, and graded; and (4) the impact of observational studies on strength of evidence (SOE) and conclusions. Study Design and SettingDescriptive study of 23 CERs published through the Effective Health Care Program of the U.S. Agency for Healthcare Research and Quality. ResultsAuthors searched for observational studies in 20 CERs, of which 18 included a median of 11 (interquartile range, 2–31) studies. Sixteen CERs incorporated the observational studies in their SOE assessments. Seventy-eight comparisons from 12 CERs included evidence from both trials and observational studies; observational studies had an impact on SOE and conclusions for 19 (24%) comparisons. There was diversity across the CERs regarding decisions to include observational studies; study designs considered; and approaches used to appraise, synthesize, and grade SOE. ConclusionReporting and methods guidance are needed to ensure clarity and consistency in how observational studies are incorporated in CERs. It was not always clear that observational studies added value in light of the additional resources needed to search for, select, appraise, and analyze such studies.

Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis.

The Cochrane Collaboration has been at the forefront of developing methods for knowledge synthesis internationally. We discuss three approaches to synthesize evidence for healthcare interventions: systematic reviews (SRs), overviews of reviews and comparative effectiveness reviews. We illustrate these approaches with examples from knowledge syntheses on interventions for bronchiolitis, a common acute paediatric condition. Some of the differences among these approaches are subtle and methods are not necessarily mutually exclusive to a single review type. Systematic reviews bring together evidence from multiple studies in a rigorous fashion for a single intervention or group of interventions. Systematic reviews, as they have developed within healthcare, often focus on single or select interventions and direct pairwise comparisons; therefore, end-users may need to read several individual SRs to inform decision making. Overviews of reviews compile information from multiple SRs relevant to a single health problem. Overviews provide the end-user with a quick overview of the available evidence; however, overviews are dependent on the methods and decisions employed at the SR level. Furthermore, overviews do not often integrate evidence from different SRs quantitatively. Comparative effectiveness reviews, as we define them here, synthesize relevant evidence from individual studies to describe the relative benefits (or harms) of a range of interventions. Comparative effectiveness reviews may use statistical methods (network meta-analysis) to incorporate direct and indirect evidence; therefore, they can provide stronger inferences about the relative effectiveness (or safety) of interventions. While potentially more expensive and time-consuming to produce, a comparative effectiveness review provides a synthesis of a range of interventions for a given condition and the relative efficacy across interventions using consistent and standardized methodology.

NLA Task Force on Statin Safety--2014 update.

1933-2874/$ see front matter 2014 http://dx.doi.org/10.1016/j.jacl.2014.03 Statins are the most widely prescribed class of medications in the United States and their benefits for lowering lowdensity lipoprotein cholesterol (LDL-C) and reducing the risk for coronary heart disease (CHD) are well documented. Statins have been the cornerstone of pharmacotherapy for the management of high blood cholesterol levels virtually since their development. The American Heart Association /American College of Cardiology 2013 guidelines recently expanded the number of individuals eligible for statin therapy by recommending it for those with: (1) clinical atherosclerotic cardiovascular disease (ASCVD), (2) LDL-C $190 mg/dL, (3) type 2 diabetes and age between 40 and 75 years with LDL-C 70 to 189 mg/dL, and (4) an estimated 10-year risk of ASCVD $7.5% and age 40 to 75 years. Considering only those who qualify according to their estimated 10-year risk of ASCVD, it has been reported that of the 101 million people in the United States age 40 to 79 years who do not have cardiovascular disease, 33 million have an estimated 10-year risk of ASCVD $7.5%, and another 13 million have a risk between 5% and 7.4%. Using crude global estimates, 920 million people worldwide would be expected to be

Clinical trial registries are of minimal use for identifying selective outcome and analysis reporting

This study aimed to examine selective outcome reporting (SOR) and selective analysis reporting (SAR) in randomized controlled trials (RCTs) and to explore the usefulness of trial registries for identifying SOR and SAR. We selected one "index outcome" for each of three comparative effectiveness reviews (CERs) of pharmacotherapy and extracted data on this outcome from trial registries and from study publications. Among 50 RCTs published since 2005 and reporting the index outcome, only 50% were listed in registries; 90% of RCTs were assessed as having SOR or SAR. The index outcome in the registry was different from that in the publication in 75% of trials in two CERs, and not specified at all in the third. Reported outcomes and analyses were not consistent between the publication's methods section and the results section in 33% and 46% of the two CERs where the index outcome was a benefit. There were no statistically significant predictors of SOR and SAR in our small sample where some predictors lacked variability. The SOR and SAR were frequent in this pilot study, and the most common type of SOR was the publication of outcomes that were not pre-specified. Trial registries were of little use in identifying SOR and of no use in identifying SAR.

Assessment of a method to detect signals for updating systematic reviews.

BackgroundSystematic reviews are a cornerstone of evidence-based medicine but are useful only if up-to-date. Methods for detecting signals of when a systematic review needs updating have face validity, but no proposed method has had an assessment of predictive validity performed.MethodsThe AHRQ Comparative Effectiveness Review program had produced 13 comparative effectiveness reviews (CERs), a subcategory of systematic reviews, by 2009, 11 of which were assessed in 2009 using a surveillance system to determine the degree to which individual conclusions were out of date and to assign a priority for updating each report. Four CERs were judged to be a high priority for updating, four CERs were judged to be medium priority for updating, and three CERs were judged to be low priority for updating. AHRQ then commissioned full update reviews for 9 of these 11 CERs. Where possible, we matched the original conclusions with their corresponding conclusions in the update reports, and compared the congruence between these pairs with our original predictions about which conclusions in each CER remained valid. We then classified the concordance of each pair as good, fair, or poor. We also made a summary determination of the priority for updating each CER based on the actual changes in conclusions in the updated report, and compared these determinations with the earlier assessments of priority.ResultsThe 9 CERs included 149 individual conclusions, 84% with matches in the update reports. Across reports, 83% of matched conclusions had good concordance, and 99% had good or fair concordance. The one instance of poor concordance was partially attributable to the publication of new evidence after the surveillance signal searches had been done. Both CERs originally judged as being low priority for updating had no substantive changes to their conclusions in the actual updated report. The agreement on overall priority for updating between prediction and actual changes to conclusions was Kappa = 0.74.ConclusionsThese results provide some support for the validity of a surveillance system for detecting signals indicating when a systematic review needs updating.

Does classroom-based Crew Resource Management training improve patient safety culture? A systematic review.

Aim:To evaluate the evidence of the effectiveness of classroom-based Crew Resource Management training on safety culture by a systematic review of literature.Methods:Studies were identified in PubMed, Cochrane Library, PsycINFO, and Educational Resources Information Center up to 19 December 2012. The Methods Guide for Comparative Effectiveness Reviews was used to assess the risk of bias in the individual studies.Results:In total, 22 manuscripts were included for review. Training settings, study designs, and evaluation methods varied widely. Most studies reporting only a selection of culture dimensions found mainly positive results, whereas studies reporting all safety culture dimensions of the particular survey found mixed results. On average, studies were at moderate risk of bias.Conclusion:Evidence of the effectiveness of Crew Resource Management training in health care on safety culture is scarce and the validity of most studies is limited. The results underline the necessity of more valid study designs, preferably using triangulation methods.

Opportunities and challenges in using studies without a control group in comparative effectiveness reviews

When examining the evidence on therapeutic interventions to answer a comparative effectiveness research question, one should consider all studies that are informative on the interventions' causal effects. "Single group studies" evaluate outcomes longitudinally in cohorts of subjects who are managed with a single treatment strategy. Because these studies are "missing" a direct, concurrent comparison group, they are typically deemed non-informative on comparative effectiveness. However, in principle, single group studies can provide information on causal treatment effects by extrapolating expected outcomes in the "missing" untreated arm. Single group studies rely on before-after, implicit, or historical comparisons as a proxy for an ideal comparison group. The validity of these comparisons must be carefully examined on a case-by-case basis. While in many cases, researchers will disagree on whether such extrapolations are reasonable; circumstances exist where such studies are generally acceptable as a source of evidence. This article provides an overview of issues related to the interpretation of single group studies with a focus on the assumptions required to support their consideration in comparative effectiveness reviews. We discuss the various settings in which single group studies are employed, common research designs that systematic reviewers need to interpret, and challenges associated with using these designs to inform comparative effectiveness questions.

PHP232 - The Expanding Scope of Comparative Effectiveness Reviews Requires Collaborative Information Systems Solutions

The scope of systematic reviews and health technology assessments is rapidly expanding due to increasing demand for more complex models that account for patient, treatment, and trial characteristics, network meta-analyses that include more interventions, and the growing number of clinical trials. By the year 2000, the effort required to publish a typical systematic review had already reached the thousands of person-hours, which were predominantly spent on data acquisition tasks. Innovative solutions are required to prevent the costs of comparative effectiveness research from ballooning out of proportion. Typically only the end product of systematic reviewing, a report summarizing the evidence, is made widely available. However, capturing the intermediate results of literature searching, publication screening, and data extraction has the potential to greatly enhance the efficiency of future reviews. In the face of the increasing scope of systematic reviews, this unnecessary duplication of effort must be eliminated. However, doing so is difficult due to the current culture of data protectionism and a lack of suitable software that enables convenient and useful sharing of the intermediate results. Building on our previously published reviews of software for systematic review and trial analytics, the talk identifies the technical and cultural challenges to be met. We propose that a web-based solution enabling the global research community to contribute their intermediate results in exchange for access to the data contributed by others could rapidly gain momentum. Such a system would challenge data protectionism by greatly reducing the level of investment required for data acquisition. A disruptive web-based system enabling a massively collaborative approach to systematic reviewing can make data protectionism obsolete and eliminate much of the effort required for future systematic reviews.