non-Cochrane Reviews Research Articles

PMU13 - Methodological Quality of Drug Safety Systematic Reviews: A Cross-Sectional Study

To examine the methodological quality of drug safety systematic reviews, and to investigate factors associated with the methodological quality of drug safety systematic review by conducting a cross-sectional survey. We searched PubMed to identity published in the Cochrane Database of Systematic Reviews and Core Clinical Journals indexed in 2015, and randomly sampled the studies with 1:1 of Cochrane and non-Cochrane systematic reviews regarding drug safety. We used the AMSTAR tool to assess the methodological quality of included studies. Multivariable linear regression analyses were used to assess the association between the seven specified variables with AMSTAR score. We included 120 systematic reviews, including 60 Cochrane systematic reviews and 60 non-Cochrane systematic reviews. The mean (standard deviation) overall score for the AMSTAR tool (11 items) in Cochrane systematic reviews was higher than in non-Cochrane systematic reviews (10±0.9 vs. 8.67±0.95; P＜0.001). Multivariable linear regression analyses showed that type of systematic review (P＜0.001) and area of diseases (P=0.01) were the factors associated with the methodological quality. Our study showed that Cochrane systematic reviews had better methodological quality than non-Cochrane systematic reviews. Type of systematic review and area of diseases were the factor associated with the reporting and methodological quality, and the methodological quality of non-Cochrane systematic review should be improved in the future.

Comparison of methodological quality rating of systematic reviews on neuropathic pain using AMSTAR and R-AMSTAR

BackgroundSystematic reviews (SRs) in the field of neuropathic pain (NeuP) are increasingly important for decision-making. However, methodological flaws in SRs can reduce the validity of conclusions. Hence, it is important to assess the methodological quality of NeuP SRs critically. Additionally, it remains unclear which assessment tool should be used. We studied the methodological quality of SRs published in the field of NeuP and compared two assessment tools.MethodsWe systematically searched 5 electronic databases to identify SRs of randomized controlled trials of interventions for NeuP available up to March 2015. Two independent reviewers assessed the methodological quality of the studies using the Assessment of Multiple Systematic Reviews (AMSTAR) and the revised AMSTAR (R-AMSTAR) tools. The scores were converted to percentiles and ranked into 4 grades to allow comparison between the two checklists. Gwet’s AC1 coefficient was used for interrater reliability assessment.ResultsThe 97 included SRs had a wide range of methodological quality scores (AMSTAR median (IQR): 6 (5–8) vs. R-AMSTAR median (IQR): 30 (26–35)). The overall agreement score between the 2 raters was 0.62 (95% CI 0.39–0.86) for AMSTAR and 0.62 (95% CI 0.53–0.70) for R-AMSTAR. The 31 Cochrane systematic reviews (CSRs) were consistently ranked higher than the 66 non-Cochrane systematic reviews (NCSRs). The analysis of individual domains showed the best compliance in a comprehensive literature search (item 3) on both checklists. The results for the domain that was the least compliant differed: conflict of interest (item 11) was the item most poorly reported on AMSTAR vs. publication bias assessment (item 10) on R-AMSTAR. A high positive correlation between the total AMSTAR and R-AMSTAR scores for all SRs, as well as for CSRs and NCSRs, was observed.ConclusionsThe methodological quality of analyzed SRs in the field of NeuP was not optimal, and CSRs had a higher quality than NCSRs. Both AMSTAR and R-AMSTAR tools produced comparable quality ratings. Our results point out to weaknesses in the methodology of existing SRs on interventions for the management NeuP and call for future improvement by better adherence to analyzed quality checklists, either AMSTAR or R-AMSTAR.

Heterogeneity in Cochrane and non-Cochrane meta-analyses in orthodontics

ObjectivesHeterogeneity describes the percentage of variability across the study effects that can be attributed to between-study differences in a meta-analysis. The aim of this project was to explore the magnitude of heterogeneity in Cochrane and non-Cochrane meta-analyses in orthodontic research and to identify possible associations between heterogeneity (I2) and a number of study characteristics including number of studies, type of outcome and type of analysis. MethodsThe contents of five major orthodontic journals and the Cochrane Database of Systematic Reviews were electronically searched from January 2000 to December 2017 to identify Systematic Reviews (SRs) with at least one meta-analysis. Included records were screened for reporting of I2 classified into four categories: 0%, 1–29%, 30–59%, 60–100%. Associations between I2 and review-level and synthesis-level characteristics were tested. Univariable and multivariable mixed effects ordinal logistic regression was used to identify significant predictors for statistical heterogeneity. ResultsA total of 72 SRs comprising 391 meta-analyses were included with the majority based on non-Cochrane reviews (n = 54, 75%). Overall, 125 meta-analyses (32%) reported heterogeneity explained by chance (I2 = 0%), whereas high values of I2 (∼60–100%) were seen in 152 syntheses (39%). In the multivariable analysis, inclusion of each additional study within the synthesis presented 20% higher odds for substantial/considerable heterogeneity compared to lower heterogeneity categories (OR = 1.20; 95%CIs: 1.09, 1.31; p < 0.001). Use of fixed effect analysis (OR = 0.25; 95%CIs: 0.12, 0.55; p = 0.001) was associated with significantly lower odds. Cochrane versus non-Cochrane meta-analyses were not associated with higher odds for substantial/considerable heterogeneity (OR = 2.81; 95%CIs: 0.53, 14.91; p = 0.22). ConclusionsSubstantial statistical heterogeneity is present within a considerable number of orthodontic meta-analyses. Further efforts should be made to improve understanding of decisions to undertake meta-analyses and selection of studies eligible for inclusion. Clinical significanceThe consistency of meta-analyses could be improved with more careful consideration of individual study characteristics. Reduced heterogeneity in meta-analyses will ensue more solid evidence based decisions for clinical practice

Elements of integrated care approaches for older people: a review of reviews

ObjectiveThe World Health Organization (WHO) recently proposed an Integrated Care for Older People approach to guide health systems and services in better supporting functional ability of older people. A knowledge...

Evaluating characteristics of PROSPERO records as predictors of eventual publication of non-Cochrane systematic reviews: a meta-epidemiological study protocol

BackgroundEpidemiology and the reporting characteristics of systematic reviews (SRs) and meta-analyses (MAs) are well known. However, no study has analyzed the influence of protocol features on the probability that a study’s results will be finally reported, thereby indirectly assessing the reporting bias of International Prospective Register of Systematic Reviews (PROSPERO) registration records.ObjectiveThe objective of this study is to explore which factors are associated with a higher probability that results derived from a non-Cochrane PROSPERO registration record for a systematic review will be finally reported as an original article in a scientific journal.Methods/designThe PROSPERO repository will be web scraped to automatically and iteratively obtain all completed non-Cochrane registration records stored from February 2011 to December 2017. Downloaded records will be screened, and those with less than 90% fulfilled or are duplicated (i.e., those sharing titles and reviewers) will be excluded. Manual and human-supervised automatic methods will be used for data extraction, depending on the data source (fields of PROSPERO registration records, bibliometric databases, etc.). Records will be classified into published, discontinued, and abandoned review subgroups. All articles derived from published reviews will be obtained through multiple parallel searches using the full protocol “title” and/or “list reviewers” in MEDLINE/PubMed databases and Google Scholar. Reviewer, author, article, and journal metadata will be obtained using different sources. R and Python programming and analysis languages will be used to describe the datasets; perform text mining, machine learning, and deep learning analyses; and visualize the data. We will report the study according to the recommendations for meta-epidemiological studies adapted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement for SRs and MAs.DiscussionThis meta-epidemiological study will explore, for the first time, characteristics of PROSPERO records that may be associated with the publication of a completed systematic review. The evidence may help to improve review workflow performance in terms of research topic selection, decision-making regarding team selection, planning relationships with funding sources, implementing literature search strategies, and efficient data extraction and analysis. We expect to make our results, datasets, and R and Python code scripts publicly available during the third quarter of 2018.

Yoga for stroke rehabilitation.

Yoga for stroke rehabilitation.

Impact of heterogeneity and effect size on the estimation of the optimal information size: analysis of recently published meta-analyses

ObjectiveTo estimate the proportion of systematic reviews that meet the optimal information size (OIS) and assess the impact heterogeneity and effect size have on the OIS estimate by type of...

Methodological Quality Assessment of Meta-Analyses of Hyperthyroidism Treatment.

Hyperthyroidism is a common condition that is associated with increased morbidity and mortality. A number of meta-analyses (MAs) have assessed the therapeutic measures for hyperthyroidism, including antithyroid drugs, surgery, and radioiodine, however, the methodological quality has not been evaluated. This study evaluated the methodological quality and summarized the evidence obtained from MAs of hyperthyroidism treatments for radioiodine, antithyroid drugs, and surgery. We searched the PubMed, EMBASE, Cochrane Library, Web of Science, and Chinese Biomedical Literature Database databases. Two investigators independently assessed the meta-analyses titles and abstracts for inclusion. Methodological quality was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. A total of 26 MAs fulfilled the inclusion criteria. Based on the AMSTAR scores, the average methodological quality was 8.31, with large variability ranging from 4 to 11. The methodological quality of English meta-analyses was better than that of Chinese meta-analyses. Cochrane reviews had better methodological quality than non-Cochrane reviews due to better study selection and data extraction, the inclusion of unpublished studies, and better reporting of study characteristics. The authors did not report conflicts of interest in 53.8% meta-analyses, and 19.2% did not report the harmful effects of treatment. Publication bias was not assessed in 38.5% of meta-analyses, and 19.2% did not report the follow-up time. Large-scale assessment of methodological quality of meta-analyses of hyperthyroidism treatment highlighted methodological strengths and weaknesses. Consideration of scientific quality when formulating conclusions should be made explicit. Future meta-analyses should improve on reporting conflict of interest.

Clinician-targeted interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care: an overview of systematic reviews.

Clinician-targeted interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care: an overview of systematic reviews.

Antibiotics for secondary prevention of coronary heart disease

Antibiotics for secondary prevention of coronary heart disease

Interventions for postoperative pain in children: An overview of systematic reviews

The aim of this study was to conduct an overview of systematic reviews that summarizes the results about efficacy and safety from randomized controlled trials involving the various strategies used for postoperative pain management in children. We searched the Cochrane Database of Systematic Reviews, CINAHL, Database of Reviews of Effect, Embase, MEDLINE, and PsycINFO from the earliest date to January 24, 2016. This overview included 45 systematic reviews that evaluated interventions for postoperative pain in children. Out of 45 systematic reviews that investigated various interventions for postoperative pain in children, 19 systematic reviews (42%) presented conclusive evidence of efficacy. Positive conclusive evidence was reported in 18 systematic reviews (40%) for the efficacy of diclofenac, ketamine, caudal analgesia, dexmedetomidine, music therapy, corticosteroid, epidural analgesia, paracetamol, and/or nonsteroidal anti-inflammatory drugs and transversus abdominis plane block. Only one systematic review reported conclusive evidence of equal efficacy that involved a comparison of dexmedetomidine vs morphine and fentanyl. Safety of interventions was reported as conclusive in 14 systematic reviews (31%), with positive conclusive evidence for dexmedetomidine, corticosteroid, epidural analgesia, transversus abdominis plane block, and clonidine. Seven systematic reviews reported equal conclusive safety for epidural infusion, diclofenac intravenous vs ketamine added to opioid analgesia, bupivacaine, ketamine, paracetamol, and dexmedetomidine vs intravenous infusions of various opioid analgesics, oral suspension and suppository of diclofenac, only opioid, normal saline, no treatment, placebo, and midazolam. Negative conclusive statement for safety was reported in one systematic review for caudal analgesia vs noncaudal regional analgesia. More than half of systematic reviews included in this overview were rated as having medium methodological quality. Of 45 included systematic reviews, 10 were Cochrane reviews and they had higher methodological quality than non-Cochrane reviews. As evidence concerning efficacy and safety is inconclusive for most of the analyzed interventions, our review points out the need for more rigorous trials concerning pain management in children.

Inter-rater reliability of AMSTAR is dependent on the pair of reviewers

BackgroundInter-rater reliability (IRR) is mainly assessed based on only two reviewers of unknown expertise. The aim of this paper is to examine differences in the IRR of the Assessment of Multiple Systematic Reviews (AMSTAR) and R(evised)-AMSTAR depending on the pair of reviewers.MethodsFive reviewers independently applied AMSTAR and R-AMSTAR to 16 systematic reviews (eight Cochrane reviews and eight non-Cochrane reviews) from the field of occupational health. Responses were dichotomized and reliability measures were calculated by applying Holsti’s method (r) and Cohen’s kappa (κ) to all potential pairs of reviewers. Given that five reviewers participated in the study, there were ten possible pairs of reviewers.ResultsInter-rater reliability varied for AMSTAR between r = 0.82 and r = 0.98 (median r = 0.88) using Holsti’s method and κ = 0.41 and κ = 0.69 (median κ = 0.52) using Cohen’s kappa and for R-AMSTAR between r = 0.77 and r = 0.89 (median r = 0.82) and κ = 0.32 and κ = 0.67 (median κ = 0.45) depending on the pair of reviewers. The same pair of reviewers yielded the highest IRR for both instruments. Pairwise Cohen’s kappa reliability measures showed a moderate correlation between AMSTAR and R-AMSTAR (Spearman’s ρ =0.50). The mean inter-rater reliability for AMSTAR was highest for item 1 (κ = 1.00) and item 5 (κ = 0.78), while lowest values were found for items 3, 8, 9 and 11, which showed only fair agreement.ConclusionsInter-rater reliability varies widely depending on the pair of reviewers. There may be some shortcomings associated with conducting reliability studies with only two reviewers. Further studies should include additional reviewers and should probably also take account of their level of expertise.

Treatments for chronic inflammatory demyelinating polyradiculoneuropathy (CIDP): an overview of systematic reviews.

Treatments for chronic inflammatory demyelinating polyradiculoneuropathy (CIDP): an overview of systematic reviews.

Additional considerations are required when preparing a protocol for a systematic review with multiple interventions

ObjectivesThe number of systematic reviews that aim to compare multiple interventions using network meta-analysis is increasing. In this study, we highlight aspects of a standard systematic review protocol that may need modification when multiple interventions are to be compared. Study Design and SettingWe take the protocol format suggested by Cochrane for a standard systematic review as our reference and compare the considerations for a pairwise review with those required for a valid comparison of multiple interventions. We suggest new sections for protocols of systematic reviews including network meta-analyses with a focus on how to evaluate their assumptions. We provide example text from published protocols to exemplify the considerations. ConclusionStandard systematic review protocols for pairwise meta-analyses need extensions to accommodate the increased complexity of network meta-analysis. Our suggested modifications are widely applicable to both Cochrane and non-Cochrane systematic reviews involving network meta-analyses.

The methodological and reporting quality of systematic reviews from China and the USA are similar

ObjectivesTo compare the methodological and reporting quality of systematic reviews by authors from China and those from the United States (USA). Study Design and SettingFrom systematic reviews of randomized trials published in 2014 in English, we randomly selected 100 from China and 100 from the USA. The methodological quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool, and reporting quality assessed using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) tool. ResultsCompared with systematic reviews from the USA, those from China were more likely to be a meta-analysis, published in low-impact journals, and a non-Cochrane review. The mean summary Assessing the Methodological Quality of Systematic Reviews score was 6.7 (95% confidence interval: 6.5, 7.0) for reviews from China and 6.6 (6.1, 7.1) for reviews from the USA, and the mean summary Preferred Reporting Items for Systematic Reviews and Meta-analyses score was 21.2 (20.7, 21.6) for reviews from China and 20.6 (19.9, 21.3) for reviews from the USA. The differences in summary quality scores between China and the USA were statistically nonsignificant after adjusting for multiple review factors. ConclusionThe overall methodological and reporting quality of systematic reviews by authors from China are similar to those from the USA, although the quality of systematic reviews from both countries could be further improved.

PP024 Changes In Reporting Characteristics Of Systematic Reviews For The United Kingdom

INTRODUCTION:A recent publication reported that increasing numbers of systematic reviews are being published and, although standards have improved, many are still poorly conducted and reported, especially non-Cochrane systematic reviews (1). The aim of this study was to assess the quality of the conduct and reporting of systematic reviews undertaken for the United Kingdom (UK) Health Technology Assessment (HTA) programme and published in the International Journal of Technology Assessment in Health Care (2) and compare those undertaken in 2004 and 2014.METHODS:A comparative sample of all systematic reviews published in 2004 and 2014 in the UK HTA monograph series was identified by a structured search of MEDLINE in August 2016. After piloting of the form, two reviewers each extracted relevant data. These data were tabulated and summarized.RESULTS:The search identified twenty-three systematic reviews from 2004 and thirty from 2014. By 2014, compared with 2004, a smaller proportion of treatment (53 percent versus 70 percent) and pharmaceutical (20 percent versus 57 percent) reviews were being published. In 2014, there were much higher percentages of review registrations (70 percent versus 0 percent) and available protocols (90 percent versus 17 percent); increased explicit inclusion of unpublished literature (65 percent versus 39 percent); less frequent use of local checklists (32 percent versus 61 percent) for critical appraisal; more complete reporting of study flow for inclusion (97 percent versus 57 percent) and exclusion (91 percent and 65 percent) of studies; and there were more reviews reporting limitations affecting the review itself (73 percent versus 49 percent). The process had clearly become more reflective and rigorous. However, some previous weaknesses persisted, including the general absence of any assessment of publication bias and the failure to report overall numbers of patients in the review.CONCLUSIONS:Marked improvements can be seen in the conduct and reporting of systematic reviews published by the UK HTA programme as a result of the publication and general acceptance of the PRISMA statement (3) and the increased application of a smaller number of relevant standards.

Fluid resuscitation for children with severe febrile illness and septic shock in resource-limited settings

Reason for withdrawal from publication The authors were unable to complete this review. A non-Cochrane review was published covering this topic (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4010848/) and therefore a Cochrane review is no longer required.

Reporting of financial and non-financial conflicts of interest by authors of systematic reviews: a methodological survey

BackgroundConflicts of interest may bias the findings of systematic reviews. The objective of this methodological survey was to assess the frequency and different types of conflicts of interest that authors...

Reliable evidence for efficacy of single dose oral analgesics.

Data sourcesThe Cochrane library was searched for Cochrane systematic reviews.Study selectionCochrane reviews on single pain medications for the treatment of acute pain were included. Non-Cochrane reviews were included for tramadol.Data extraction and synthesisTwo reviewers independently searched, selected reviews for inclusion, assessed quality and performed data extraction. A protocol in case of disagreement was in place. Data were collected on number of included studies and participants, drug, dose and formulation and pain model. The authors concentrated on the amount of information and the potential for publication bias.Pain relief was calculated using at least 50% maximum pain relief, as a percentage, and as NNTs. Duration of analgesia was measured as mean or median and time to remedication was calculated as percentage of patients.ResultsThirty-nine reviews including 41 interventions were analysed and NNTs for at least 50% maximum pain relief were summarised in a graphic. NNTs range from almost one all the way to five. Only one intervention, codeine 60, had an NNT ≥10. Results judged to be reliable were listed in detail. Mean or median time to remedication was also presented in a graphic.The authors conclude that there is a great amount of quality information on single dose analgesics, and highlighted the potential benefit of fast acting formulations and fixed formulations to achieve good long-lasting analgesia.ConclusionsThere is a wealth of reliable evidence on the analgesic efficacy of single dose oral analgesics. Fast acting formulations and fixed dose combinations of analgesics can produce good and often long-lasting analgesia at relatively low doses. There is also important information on drugs for which there are no data, inadequate data, or where results are unreliable due to susceptibility to publication bias. This should inform choices by professionals and consumers.

Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study.

BackgroundSystematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs.Methods and FindingsWe searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics.ConclusionsAn increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.