non-Cochrane Systematic Reviews Research Articles

Authorship diversity among systematic reviews in eyes and vision

ImportanceThe inclusion of authors from diverse backgrounds and with different lived experiences is critical to ensuring the questions addressed in systematic reviews (SRs), as well as the subsequent conclusions and recommendations made, are representative of the global community.ObjectiveTo assess the gender and geographic diversity of authors among all Cochrane SRs in eyes and vision as compared with a random sample of non-Cochrane SRs of interventions in the field of eyes and vision.DesignThe Cochrane Eyes and Vision US Satellite maintains a database of SRs in the field of eyes and vision. We selected all (n = 313) eyes and vision intervention SRs published in The Cochrane Library and a random sample of 313 eyes and vision intervention SRs published elsewhere for this study. We determined gender of the first and corresponding authors (“woman,” “man,” or “unknown”) using a previously developed algorithm and their location based on institution country and the World Health Organization region.ResultsFrom the 626 reviews included in our sample, we identified 751 unique authors who comprised 887 author positions (i.e., first and/or corresponding authors). We were able to ascertain the gender of 647/751 (86%) authors: 276 women and 371 men. Among Cochrane eyes and vision SRs, the proportions of women in first and/or corresponding author positions were consistent and approximately equal to men. Among non-Cochrane eyes and vision SRs, the representation of women was markedly lower as corresponding authors than other positions. Most authors of Cochrane eyes and vision SRs were from the UK (31%) and USA (26%), whereas most authors of non-Cochrane SRs were from China (34%).Conclusions and relevanceCompared with authors of non-Cochrane SRs in eyes and vision, authors of Cochrane SRs appear to have approximately equal representation of women and men among perceived important author positions and be located in European and North American countries, possibly due to the locations of the Cochrane editorial teams. Cochrane Eyes and Vision should continue to recruit authors from around the world in locations that reflect the global burden of eye disease.

Methodological Quality Analysis of Systematic Review for the Treatment of Rotator Cuff Disease.

Objective To evaluate the methodological quality of systematic reviews for the surgical and nonsurgical treatment of individuals with rotator cuff syndrome; to compare, through the Assessing the methodological quality of systematic reviews (AMSTAR) instrument, the quality of studies found in the Cochrane Library, PubMed (Publisher Medline ), EMBASE andQinsightdatabases. Methods This is a descriptive and comparative cross-sectional study, in which two independent authors analyzed, through the AMSTAR instrument, the methodological quality of Cochrane and non-Cochrane systematic reviews on the treatment of individuals diagnosed with rotator cuff syndrome. Results A total of 76 systematic reviews were evaluated by the AMSTAR instrument. The overall mean score was 6.1 (±2.1) and the mean per database was 9.1 (±0.9) for the Cochrane reviews and 5.7 (±1.8) for the non-Cochrane reviews. The lowest-scoring item of AMSTAR was 11, related to the display of the conflict of interests of the publication. In a comparative analysis of the final variable score, there was a statistical difference between the Cochrane and non-Cochrane studies. Conclusion According to the present study, systematic reviews using the Cochrane methodology have a better methodological quality compared to non-Cochrane studies on the treatment of rotator cuff dysfunctions.

Cochrane risk of bias tool was used inadequately in the majority of non-Cochrane systematic reviews

ObjectivesTo analyze how many non-Cochrane systematic reviews (NCSRs) used Cochrane's risk of bias (RoB) tool, domains they used, and whether judgments and comments about RoB were in line with Cochrane Handbook. MethodsThis was a methodological (research-on-research) study. We retrieved NCSRs from PubMed, extracted information about methods used for RoB assessment, and if they used 2011 Cochrane RoB tool, we analyzed their RoB methods and compared them with Cochrane Handbook guidance. ResultsWe included 508 NCSRs; 431 (85%) reported they analyzed RoB, and 269 (53%) used Cochrane RoB tool. Only 16 of those 269 (5.9%) reported both a judgment and a supporting comment in the Cochrane RoB table in the manuscript (N = 4) or in a supplementary file (N = 12). Fifteen reviews, with 158 included trials, used judgments low/high/unclear; 41% of analyzed available judgments were inadequate, either because judgment was not in line with comment or comment was missing. ConclusionsMost NCSRs use Cochrane RoB tool to assess RoB, but most of them reported it incompletely, with high prevalence of inadequate judgments. Authors, editors, and peer-reviewers should make an effort to improve completeness and adequacy of Cochrane RoB assessment in non-Cochrane reviews.

Searching practices and inclusion of unpublished studies in systematic reviews of diagnostic accuracy.

IntroductionMany diagnostic accuracy studies are never reported in full in a peer‐reviewed journal. Searching for unpublished studies may avoid bias due to selective publication, enrich the power of systematic reviews, and thereby help to reduce research waste. We assessed searching practices among recent systematic reviews of diagnostic accuracy.MethodsWe extracted data from 100 non‐Cochrane systematic reviews of diagnostic accuracy indexed in MEDLINE and published between October 2017 and January 2018 and from all 100 Cochrane systematic reviews of diagnostic accuracy published by December 2018, irrespective of whether meta‐analysis had been performed.ResultsNon‐Cochrane and Cochrane reviews searched a median of 4 (IQR 3‐5) and 6 (IQR 5‐9) databases, respectively; most often MEDLINE/PubMed (n = 100 and n = 100) and EMBASE (n = 81 and n = 100). Additional efforts to identify studies beyond searching bibliographic databases were performed in 76 and 98 reviews, most often through screening reference lists (n = 71 and n = 96), review/guideline articles (n = 18 and n = 52), or citing articles (n = 3 and n = 42). Specific sources of unpublished studies were searched in 22 and 68 reviews, for example, conference proceedings (n = 4 and n = 18), databases only containing conference abstracts (n = 2 and n = 33), or trial registries (n = 12 and n = 39). At least one unpublished study was included in 17 and 23 reviews. Overall, 39 of 2082 studies (1.9%) included in non‐Cochrane reviews were unpublished, and 64 of 2780 studies (2.3%) in Cochrane reviews, most often conference abstracts (97/103).ConclusionSearching practices vary considerably across systematic reviews of diagnostic accuracy. Unpublished studies are a minimal fraction of the evidence included in recent reviews.

Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status

ObjectivesThe aim of this study was to analyze the publication path of non-Cochrane systematic reviews (SR) in the field of anesthesiology and pain registered in the PROSPERO database. Study Design and SettingWe analyzed characteristics of SRs registered in PROSPERO from its inception to May 2017 and their publication status in August 2018. We surveyed corresponding authors of unpublished SRs about accuracy of PROSPERO status and reasons for nonpublication. ResultsAfter screening 1,408 records from PROSPERO database, we found that the majority had “ongoing” (76.3%) and “completed not published” (9.2%) status. Survey of authors showed that most of the records had not been updated (82.4%, 526/638); SR had already been published in 75.2% (396/526), and work on SR had been discontinued in 7.8% (41/526) of cases. In total, based on PROSPERO status, survey of authors, and database searches, 53.6% (742/1,384) of SRs had been published within a period of 1.3 years or more following their registration. Main reasons for discontinuing work on SR were publication of an SR with similar or same topic by another author team and rejection of SR manuscript. ConclusionOnly 16.3% of PROSPERO records had accurate status, and 46.4% of SRs were still unpublished. Further steps to ensure accuracy of PROSPERO status are needed, along with developing strategies for improvement of SR production process.

Characteristics of stakeholder involvement in systematic and rapid reviews: a methodological review in the area of health services research

ObjectiveEngaging stakeholders in reviews is considered to generate more relevant evidence and to facilitate dissemination and use. As little is known about stakeholder involvement, we assessed the characteristics of their...

Rehabilitation interventions for improving balance following stroke: An overview of systematic reviews.

BackgroundThe aim of this study was to synthesize evidence from systematic reviews, to summarise the effects of rehabilitation interventions for improving balance in stroke survivors.MethodsWe conducted an overview of systematic reviews (SRs). We included Cochrane Systematic Reviews and non-Cochrane Systematic Reviews of randomized-controlled clinical trials and not-randomized clinical trials, in all types of stroke, comparing the effects of interventions, control interventions and no interventions on balance-related outcomes. We conducted a comprehensive search of electronic databases, from inception to December 2017. Data extracted included: number and type of participants, type of intervention, control intervention, method of assessing risk of bias of primary studies, balance outcome measures and results of statistical meta-analyses. Methodological quality of included reviews was assessed using AMSTAR 2. A narrative description of the characteristics of the SRs was provided and results of meta-analyses summarised with reference to their methodological quality.Results51 SRs (248 primary studies and 10,638 participants) met the inclusion criteria and were included in the overview. All participants were adults with stroke. A wide variety of different balance and postural control outcomes were included. 61% of SRs focussed on the effectiveness of physical therapy, 20% virtual reality, 6% electromechanical devices, 4% Tai-Chi, whole body vibration and circuit training intervention, and 2% cognitive rehabilitation. The methodology of 54% of SRs were judged to be of a “low or critically low” quality, 23% “moderate” quality and 22% “high” quality.ConclusionsThere are 51 SRs of evidence relating to the effectiveness of interventions to improve balance in people with stroke, but the majority of these are of poor methodological quality, limiting our ability to draw clear implications. Only 22% of these SRs were judged to be of high quality, highlighting the need to address important methodological issues within rehabilitation research.

Poor Quality in Systematic Reviews on PTSD and EMDR - An Examination of Search Methodology and Reporting.

Background: Different user groups regard systematic reviews as reliable and valuable sources for answering research questions. For systematic reviews to fulfill their purpose, methodological quality in all stages are of importance. The studies identified in a systematic search form the basis of the review, thus the search process methodology is important for both performing and reporting the search. The purpose of the present study was to evaluate the quality of non-Cochrane systematic reviews by analyzing how they perform and report the search. This is exemplified by systematic reviews on eye movement desensitization and reprocessing (EMDR), a trauma-focused therapy commonly used for post-traumatic stress disorder (PTSD).Methods and Results: We examined the method chapters of 20 systematic reviews on the subject, and rated their searches and reporting using relevant elements from the Cochrane Handbook and PRISMA. We found inadequacies in the methods employed for searching and reporting the search strategy, which could have been avoided by greater adherence to guiding documents for performing systematic reviews.Conclusion: Our findings raise important questions for future debate on the risk of omitting studies, thus impairing the conclusions in a systematic review. For clinical purposes, researchers should investigate if, and how, the search strategy in a systematic review affects the body of knowledge and the results.

Statistical significance did not affect time to publication in non-Cochrane systematic reviews: a metaepidemiological study

ObjectivesThe objective of the study was to investigate time-lag bias based on statistical significance in findings in non-Cochrane systematic reviews (SRs) and explore barriers to publication for review authors. Study Design and SettingWe included SRs of randomized controlled trials with completed analyses for protocols registered in the international prospective register of systematic reviews (PROSPERO) by the end of 2014. To obtain unpublished information, we sent questionnaires regarding barriers to publication of SRs to all authors between May and November 2018 and examined the association between the statistical significance of SR findings and time to acceptance. ResultsAmong the 241 SR authors contacted, 141 (58%) responded, of whom 103 met the eligibility criteria and agreed to participate. Ninety-three (90%) of the protocols had been published as journal articles and 11 (11%) remained unpublished. Statistical significance was not significantly associated with time to acceptance (adjusted hazard ratio, 1.36; 95% confidence interval, 0.84 to 2.19). The authors reported lack of time (52%), rejection by journals (42%), duplication of similar topics (42%), and nonsignificant results (29%) as barriers to publication of SRs. ConclusionWe found no convincing evidence of time-lag bias based on statistical significance in the findings among non-Cochrane SRs registered in the PROSPERO.

Evolution of international collaborative research efforts to develop non-Cochrane systematic reviews.

This research-on-research study describes efforts to develop non-Cochrane systematic reviews (SRs) by analyzing demographical and time-course collaborations between international institutions using protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO) or published in scientific journals. We have published an a priori protocol to develop this study. Protocols published in scientific journals were searched using the MEDLINE and Embase databases; the query terms “Systematic review” [Title] AND “protocol” [Title] were searched from February 2011 to December 2017. Protocols registered at PROSPERO during the same period were obtained by web scraping all non-Cochrane records with a Python script. After excluding protocols that had a fulfillment or duplication rate of less than 90%, they were classified as published “only in PROSPERO”, “only in journals”, or in “journals and PROSPERO”. Results of data and metadata extraction using text mining processes were curated by two reviewers. These Datasets and R scripts are freely available to facilitate reproducibility. We obtained 20,814 protocols of non-Cochrane SRs. While “unique protocols” by reviewers’ institutions from 60 countries were the most frequent, a median of 6 (2-150) institutions from 130 different countries were involved in the preparation of “collaborative protocols”. The highest Ranked countries involved in overall protocol production were the UK, the U.S., Australia, Brazil, China, Canada, the Netherlands, Germany, and Italy. Most protocols were registered only in PROSPERO. However, the number of protocols published in scientific journals (924) or in both PROSPERO and journals (807) has increased over the last three years. Syst Rev and BMJ Open published more than half of the total protocols. While the more productive countries were involved in “unique” and “collaborative protocols”, less productive countries only participated in “collaborative protocols” that were mainly published in PROSPERO. Our results suggest that, although most countries were involved in solitary production of protocols for non-Cochrane SRs during the study period, it would be useful to develop new strategies to promote international collaborations, especially with less productive countries.

The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions

BackgroundOverviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. Overviews may identify multiple SRs that examine the same intervention for the same condition and include some, but not all, of the same primary studies. There is currently limited guidance on whether and how to include these overlapping SRs in overviews. Our objectives were to assess how different inclusion decisions in overviews of healthcare interventions affect their comprehensiveness and results, and document challenges encountered when making different inclusion decisions in overviews.MethodsWe used five inclusion decisions to conduct overviews across seven topic areas, resulting in 35 overviews. The inclusion decisions were (1) include all Cochrane and non-Cochrane SRs, (2) include only Cochrane SRs, or consider all Cochrane and non-Cochrane SRs but include only non-overlapping SRs, and in the case of overlapping SRs, select (3) the Cochrane SR, (4) the most recent SR (by publication or search date), or (5) the highest quality SR (assessed using AMSTAR). For each topic area and inclusion scenario, we documented the amount of outcome data lost and changed and the challenges involved.ResultsWhen conducting overviews, including only Cochrane SRs, instead of all SRs, often led to loss/change of outcome data (median 31% of outcomes lost/changed; range 0–100%). Considering all Cochrane and non-Cochrane SRs but including only non-overlapping SRs and selecting the Cochrane SR for groups of overlapping SRs (instead of the most recent or highest quality SRs) allowed the most outcome data to be recaptured (median 42% of lost/changed outcome recaptured; range 28–86%). Across all inclusion scenarios, challenges were encountered when extracting data from overlapping SRs.ConclusionsOverlapping SRs present a methodological challenge for overview authors. This study demonstrates that different inclusion decisions affect the comprehensiveness and results of overviews in different ways, depending in part on whether Cochrane SRs examine all intervention comparisons relevant to the overview. Study results were used to develop an evidence-based decision tool that provides practical guidance for overview authors.

A contextualisation approach to health promotion guideline development in South Africa.

Risk factors for chronic illness contribute significantly to the disease burden in South Africa. The National Department of Health (NDoH) commissioned the development of a toolkit of health promotion guidelines for use by healthcare professionals working in the primary care setting to address this burden. To (i) demonstrate the contextualisation approach to evidence-based health promotion recommendations; and (ii) present the development process of a contextually sensitive and illustrated fit-for-purpose product. A contextualised approach was used whereby evidence from rigorous guidelines produced elsewhere was tailored to local conditions. The scope of the toolkit included five risk factors and 22 conditions identified by the NDoH and was underpinned by the Theory of Planned Behaviour. Potential health promotion messages relevant to risks, conditions or both were formulated as population, intervention, comparison and outcome (PICO) questions. The team searched for and selected evidence for each PICO question in a stepwise hierarchical manner and categorised sources as: (i) World Health Organization (WHO) guidelines; (ii) Cochrane systematic reviews; and (iii) non-Cochrane systematic reviews. Those messages supported by source-based evidence were included in the toolkit with culturally appropriate illustrations. Regular engagement with stakeholders included an initial health department stakeholder consultation, a focus group with national programme managers on the appearance and content of a draft toolkit, and a presentation of the final draft at a forum of provincial managers. Final approval of the toolkit rested with programme representatives. A total of 152 PICO questions were formulated. Supporting evidence was identified from 42 current WHO guidelines and 45 Cochrane systematic reviews to answer 147 PICO questions with several guidelines relevant to more than one risk or condition. Evidence for a further five PICO questions was obtained from non-Cochrane systematic reviews. Six additional service delivery messages and four 'no harm' messages were included to align the toolkit with current national guidelines. The illustrated toolkit was well received by stakeholders nationally and provincially, with programme managers expressing a high degree of willingness to adopt a preventive approach in the primary care clinic setting. Use of a tailored contextualised approach to health promotion guidelines resulted in a culturally appropriate tool based on evidence gathered from rigorous sources and probably reduced development time and costs. Adherence to a robust framework to identify evidence ensured that the toolkit conforms to international guideline development standards.

Prophylactic management of postpartum haemorrhage in the third stage of labour: an overview of systematic reviews

BackgroundPostpartum haemorrhage is a direct cause of maternal death worldwide and usually occurs during the third stage of labour. Most women receive some type of prophylactic management, which may include pharmacological or non-pharmacological interventions. The objective of this study was to summarize systematic reviews that assessed the effects of postpartum haemorrhage prophylactic management during the third stage of labour.MethodsWe applied the guidelines for conducting an overview of reviews from the Cochrane Handbook for Systematic Reviews of Interventions. We searched MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews to identify all relevant systematic reviews of randomized controlled trials of prophylactic management of postpartum haemorrhage in the third stage of labour compared with no treatment, placebo, or another management technique. Two review authors independently extracted data and assessed methodological quality using a measurement tool to assess reviews and quality of evidence using the Grades of Recommendation, Assessment, Development, and Evaluation for primary outcomes, summarizing results narratively.ResultsWe identified 29 systematic reviews: 18 Cochrane and 11 non-Cochrane. Cochrane systematic reviews were high quality, while the quality of non-Cochrane systematic reviews varied. The following techniques suggested effective, third-stage interventions to reduce the incidence of severe postpartum haemorrhage: active management of the third stage of labour compared to physiological management, active management compared to expectant management, administration of oxytocin compared to placebo, and use of tranexamic acid compared to placebo. The following third-stage management approaches reduced the need for blood transfusion: active management compared to physiological management, active management compared to expectant management, oral misoprostol compared to placebo, and tranexamic acid compared to placebo.ConclusionsNo effective prophylactic management techniques were identified for maternal mortality. Most methods of effective prophylactic management of postpartum haemorrhage were supported by evidence; however, they were limited to low- or moderate-quality evidence, and high-quality studies are therefore needed. Outcome measures of the included systematic reviews varied. It is recommended that outcome measures in preventive postpartum haemorrhage intervention trials align with the World Health Organization guidelines.Systematic review registrationPROSPERO: CRD42016049220.

A descriptive analysis of non-Cochrane child-relevant systematic reviews published in 2014

BackgroundConsumers, clinicians, policymakers and researchers require high quality evidence to guide decision-making in child health. Though Cochrane systematic reviews (SRs) are a well-established source of evidence, little is known about the characteristics of non-Cochrane child-relevant SRs. To complement published descriptions of Cochrane SRs, we aimed to characterize the epidemiologic, methodological, and reporting qualities of non-Cochrane child-relevant SRs published in 2014.MethodsEnglish-language child-relevant SRs of quantitative primary research published outside the Cochrane Library in 2014 were eligible for this descriptive analysis. A research librarian searched MEDLINE, CINAHL, Web of Science, and PubMed in August 2015. A single reviewer screened articles for inclusion; a second verified the excluded studies. Reviewers extracted: general characteristics of the review; included study characteristics; methodological approaches. We performed univariate analyses and presented the findings narratively.ResultsWe identified 1598 child-relevant SRs containing a median (IQR) 19 (11, 33) studies. These originated primarily from high-income countries (n = 1247, 78.0%) and spanned 47 of the 53 Cochrane Review Groups. Most synthesized therapeutic (n = 753, 47.1%) or epidemiologic (n = 701, 43.8%) evidence. Though 39.3% (n = 628) of SRs included evidence related to children only, few were published in pediatric-specific journals (n = 283, 17.7%). Reporting quality seemed poor based on the items we assessed; few reviews mentioned an a-priori protocol (n = 246, 15.4%) or registration (n = 111, 6.9%), and only 23.4% (n = 374) specified a primary outcome. Many SRs relied solely on evidence from non-RCTs (n = 796, 49.8%). Less than two-thirds (n = 953, 59.6%) appraised the quality of included studies and assessments of the certainty of the body of evidence were rare (n = 102, 6.4%).ConclusionsChild-relevant Cochrane SRs are a known source of high quality evidence in pediatrics. There exists, however, an abundance of evidence from non-Cochrane SRs that may be complementary. Our findings show that high-quality non-Cochrane SRs may not be practical nor easy for knowledge users to find. Improvements are needed to ensure that evidence syntheses published outside of the Cochrane Library adhere to the high standard of conduct and reporting characteristic of Cochrane SRs.

The reporting of safety among drug systematic reviews was poor before the implementation of the PRISMA harms checklist

ObjectivesTo examine, through a cross-sectional survey, how well safety information was reported among drug systematic reviews predating PRISMA harms checklist and explore factors associated with better reporting. Study Design and SettingWe searched PubMed to identify all systematic reviews published in the Cochrane Database of Systematic Review or the core clinical journals in 2015, one year before the PRISMA harms checklist was published. We randomly selected, in a 1:1 ratio, Cochrane and non-Cochrane systematic reviews assessing drug effects (including both efficacy and safety). We used the PRISMA harms checklist published in 2016 to assess the quality of reporting of drug safety information. Multivariable linear regression analyses were used to explore the association of six prespecified variables with more complete reporting of PRISMA harms items. ResultsWe included 120 systematic reviews, including 60 Cochrane and 60 non-Cochrane reviews. Scores on the PRISMA harms checklist (23 items) were low (median 4, [first, third quartile: 2, 6]), with no difference between Cochrane and non-Cochrane reviews (4.5 [2, 7] vs. 4 [2.5, 5]; P = 0.29). Among all eligible reviews, only one item (i.e., state conclusions in coherence with the review findings) was reported adequately (proportion of adherence 81.6%); proportion of reporting for other items ranged from 1.7% to 68.3%. The four essential reporting items from PRISMA harms checklist were also poorly complied (proportion of adherence ranged from 1.7% to 9.2%). Multivariable linear regression analyses found no significant associations between any study characteristic and reporting on the PRISMA harms, likely because of limited variability in scores across studies. ConclusionsThe reporting of safety information was poor both for Cochrane and non-Cochrane drug systematic reviews predating PRISMA harms checklist. The findings suggested a strong need to use the PRISMA harms checklist for reporting safety among drug systematic reviews.

PMU13 - Methodological Quality of Drug Safety Systematic Reviews: A Cross-Sectional Study

To examine the methodological quality of drug safety systematic reviews, and to investigate factors associated with the methodological quality of drug safety systematic review by conducting a cross-sectional survey. We searched PubMed to identity published in the Cochrane Database of Systematic Reviews and Core Clinical Journals indexed in 2015, and randomly sampled the studies with 1:1 of Cochrane and non-Cochrane systematic reviews regarding drug safety. We used the AMSTAR tool to assess the methodological quality of included studies. Multivariable linear regression analyses were used to assess the association between the seven specified variables with AMSTAR score. We included 120 systematic reviews, including 60 Cochrane systematic reviews and 60 non-Cochrane systematic reviews. The mean (standard deviation) overall score for the AMSTAR tool (11 items) in Cochrane systematic reviews was higher than in non-Cochrane systematic reviews (10±0.9 vs. 8.67±0.95; P＜0.001). Multivariable linear regression analyses showed that type of systematic review (P＜0.001) and area of diseases (P=0.01) were the factors associated with the methodological quality. Our study showed that Cochrane systematic reviews had better methodological quality than non-Cochrane systematic reviews. Type of systematic review and area of diseases were the factor associated with the reporting and methodological quality, and the methodological quality of non-Cochrane systematic review should be improved in the future.

Comparison of methodological quality rating of systematic reviews on neuropathic pain using AMSTAR and R-AMSTAR

BackgroundSystematic reviews (SRs) in the field of neuropathic pain (NeuP) are increasingly important for decision-making. However, methodological flaws in SRs can reduce the validity of conclusions. Hence, it is important to assess the methodological quality of NeuP SRs critically. Additionally, it remains unclear which assessment tool should be used. We studied the methodological quality of SRs published in the field of NeuP and compared two assessment tools.MethodsWe systematically searched 5 electronic databases to identify SRs of randomized controlled trials of interventions for NeuP available up to March 2015. Two independent reviewers assessed the methodological quality of the studies using the Assessment of Multiple Systematic Reviews (AMSTAR) and the revised AMSTAR (R-AMSTAR) tools. The scores were converted to percentiles and ranked into 4 grades to allow comparison between the two checklists. Gwet’s AC1 coefficient was used for interrater reliability assessment.ResultsThe 97 included SRs had a wide range of methodological quality scores (AMSTAR median (IQR): 6 (5–8) vs. R-AMSTAR median (IQR): 30 (26–35)). The overall agreement score between the 2 raters was 0.62 (95% CI 0.39–0.86) for AMSTAR and 0.62 (95% CI 0.53–0.70) for R-AMSTAR. The 31 Cochrane systematic reviews (CSRs) were consistently ranked higher than the 66 non-Cochrane systematic reviews (NCSRs). The analysis of individual domains showed the best compliance in a comprehensive literature search (item 3) on both checklists. The results for the domain that was the least compliant differed: conflict of interest (item 11) was the item most poorly reported on AMSTAR vs. publication bias assessment (item 10) on R-AMSTAR. A high positive correlation between the total AMSTAR and R-AMSTAR scores for all SRs, as well as for CSRs and NCSRs, was observed.ConclusionsThe methodological quality of analyzed SRs in the field of NeuP was not optimal, and CSRs had a higher quality than NCSRs. Both AMSTAR and R-AMSTAR tools produced comparable quality ratings. Our results point out to weaknesses in the methodology of existing SRs on interventions for the management NeuP and call for future improvement by better adherence to analyzed quality checklists, either AMSTAR or R-AMSTAR.

Evaluating characteristics of PROSPERO records as predictors of eventual publication of non-Cochrane systematic reviews: a meta-epidemiological study protocol

BackgroundEpidemiology and the reporting characteristics of systematic reviews (SRs) and meta-analyses (MAs) are well known. However, no study has analyzed the influence of protocol features on the probability that a study’s results will be finally reported, thereby indirectly assessing the reporting bias of International Prospective Register of Systematic Reviews (PROSPERO) registration records.ObjectiveThe objective of this study is to explore which factors are associated with a higher probability that results derived from a non-Cochrane PROSPERO registration record for a systematic review will be finally reported as an original article in a scientific journal.Methods/designThe PROSPERO repository will be web scraped to automatically and iteratively obtain all completed non-Cochrane registration records stored from February 2011 to December 2017. Downloaded records will be screened, and those with less than 90% fulfilled or are duplicated (i.e., those sharing titles and reviewers) will be excluded. Manual and human-supervised automatic methods will be used for data extraction, depending on the data source (fields of PROSPERO registration records, bibliometric databases, etc.). Records will be classified into published, discontinued, and abandoned review subgroups. All articles derived from published reviews will be obtained through multiple parallel searches using the full protocol “title” and/or “list reviewers” in MEDLINE/PubMed databases and Google Scholar. Reviewer, author, article, and journal metadata will be obtained using different sources. R and Python programming and analysis languages will be used to describe the datasets; perform text mining, machine learning, and deep learning analyses; and visualize the data. We will report the study according to the recommendations for meta-epidemiological studies adapted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement for SRs and MAs.DiscussionThis meta-epidemiological study will explore, for the first time, characteristics of PROSPERO records that may be associated with the publication of a completed systematic review. The evidence may help to improve review workflow performance in terms of research topic selection, decision-making regarding team selection, planning relationships with funding sources, implementing literature search strategies, and efficient data extraction and analysis. We expect to make our results, datasets, and R and Python code scripts publicly available during the third quarter of 2018.

Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis

ObjectivesThe objective of the study was to investigate the application and interpretation of statistical analyses in a cross-section of systematic reviews (SRs) of therapeutic interventions, without restriction by journal, clinical condition, or specialty. Study Design and SettingWe evaluated a random sample of SRs assembled previously, which were indexed in MEDLINE® during February 2014, focused on a treatment or prevention question, and reported at least one meta-analysis. The reported statistical methods used in each SR were extracted from articles and online appendices by one author, with a 20% random sample extracted in duplicate. ResultsWe evaluated 110 SRs; 78/110 (71%) were non-Cochrane SRs and 55/110 (50%) investigated a pharmacological intervention. The SRs presented a median of 13 (interquartile range: 5–27) meta-analytic effects. When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs; however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR. ConclusionsThere is scope for improvement in the application and interpretation of statistical analyses in SRs of therapeutic interventions. The involvement of statisticians on the SR team and establishment of partnerships between researchers with specialist expertise in SR methods and journal editors may help overcome these shortcomings.

Treatments for chronic inflammatory demyelinating polyradiculoneuropathy (CIDP): an overview of systematic reviews.

Treatments for chronic inflammatory demyelinating polyradiculoneuropathy (CIDP): an overview of systematic reviews.