A Critical Appraisal of Reporting Bias in Systematic Reviews and Meta-Analyses Addressing Knee Chondral Defects.

This study aimed to compare the methodological quality and risk bias of orthodontic systematic reviews (SRs) using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) and ROBIS (Risk of Bias in Systematic Review) tools. A search of electronic databases (OVID and Medline) was undertaken to identify orthodontic SRs published in five primary orthodontic journals (January 2015 to December 2018) and the Cochrane Library of Systematic Reviews (January 2000 to January 2018). Full articles were reviewed by two assessors against the eligibility criteria. Methodological quality of each SR was gauged using the AMSTAR tool with a score of 0 or 1 given for each of the 11 items. Cumulative totals were calculated and scores between 4 and 8 represented poor to fair methodological quality and 9 or greater deemed to be good. As per the ROBIS tool, the risk of bias (ROB) for each domain was assessed and the overall ROB was classified as low, high, or unclear. A total of 91 SRs were included. The median AMSTAR score was 8 (IQR = 3). The methodological quality of 47.3% SRs was rated good. SRs without protocol registration (Coef: -3.00, 95% CI: -3.72, -2.28, P < 0.001) and American continent SRs (Coef: -1.00, 95% CI: -1.72, -0.21, P = 0.007) were associated with lower AMSTAR scores. A total of 56.0% SRs were rated a low ROB, with a lower ROB apparent in multicentre SRs (OR: 0.27, 95% CI: 0.11, 0.64, P = 0.003) and a higher ROB evident in SRs without a registered protocol (OR: 111.81, 95% CI: 22.34, 559.62, P < 0.001). When adjusted for the effect of AMSTAR score on ROB, a higher ROB was associated with SRs without protocol registration (OR: 32.24, 95% CI: 6.03, 172.44, P ≤ 0.001). As the AMSTAR score (per unit) increased, the odds of having a high ROB rating decreased (OR: 0.31, 95% CI: 0.21, 0.45, P ≤ 0.001). As the methodological quality rating of orthodontic SRs increases, a reduction in the ROB is evident.

To evaluate the methodological quality of recent systematic reviews of interventions for children with cerebral palsy in order to determine the level of confidence in the reviews' conclusions. A comprehensive search of 22 databases identified eligible systematic reviews with and without meta-analysis published worldwide from 2015 to 2019. We independently extracted data and used A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2) to appraise methodological quality. Eighty-three systematic reviews met strict eligibility criteria. Most were from Europe and Latin America and reported on rehabilitative interventions. AMSTAR-2 appraisal found critically low confidence in 88% (n=73) because of multiple and varied deficiencies. Only 7% (n=6) had no AMSTAR-2 critical domain deficiency. The number of systematic reviews increased fivefold from 2015 to 2019; however, quality did not improve over time. Most of these systematic reviews are considered unreliable according to AMSTAR-2. Current recommendations for treating children with CP based on these flawed systematic reviews need re-evaluation. Findings are comparable to reports from other areas of medicine, despite the general perception that systematic reviews are high-level evidence. The required use of current widely accepted guidance for conducting and reporting systematic reviews by authors, peer reviewers, and editors is critical to ensure reliable, unbiased, and transparent systematic reviews. What this paper adds Confidence was critically low in the conclusions of 88% of systematic reviews about interventions for children with cerebral palsy (CP). Quality issues in the sample were not limited to systematic reviews of non-randomized trials, or to those about certain populations of CP or interventions. The inclusion of meta-analysis did not improve the level of confidence in these systematic reviews. Numbers of systematic reviews on this topic increased over the 5 search years but their methodological quality did not improve.

To evaluate the reporting and methodological quality of relevant systematic reviews (SRs) and meta-analyses (MAs) onDipeptidyl peptidase-4 inhibitors (DPP-4I) forType 2 diabetes mellitus (T2DM). Relevant SRs and MAs on T2DM and DPP-4I published between 2017 and November 2021 were retrieved from PubMed, Embase, Cochrane Library, Web of Science, VIP, CNKI, CBM, and WanFang databases. Two independent reviewers performed the search, selection, and data extraction. The reporting and methodological quality of the reviewers was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR 2) tools. The relationship between reporting and methodological quality score was assessed with the Spearman correlation test. Twenty-one studies involving 151,715 participants were included in the study. This overview showed that DPP-4I was safer and more efficacious than other anti-hyperglycemic drugs (OADs) in treating T2DM. The methodological quality of one SR was low, while the rest were very low. Thus, refinements are needed in the quality of protocol and registration information, a complete search strategy, the summary of the evidence, the listing of excluded studies, assessing the potential impact of risk of bias in RCTs, and discussing the RoB on MA results, and the funding of RCTs need improvement for generating SR. In addition, the reporting and methodological quality scores were moderately correlated (rS = 0.66, P = 0.001). DPP-4I is safer and more efficacious than OADs in treating T2DM. However, the reporting and methodological quality of the related SRs was unsatisfactory. Therefore, PRISMA and AMSTAR 2 analyses should be followed to enhance the overall quality of future SRs.

Effectiveness and safety of acupuncture for children with cerebral palsy: An overview of systematic reviews

The purpose of the present study was to evaluate the methodological quality and risk of bias in systematic reviews (SRs) on the effectiveness of peri-implantitis treatments. We searched four electronic databases: MEDLINE, Web of Science, Cochrane Database of Systematic Reviews, and EMBASE. Previous SRs focusing on peri-implantitis treatment published between 2010 and 2017 were identified. After literature screening, eligible SRs were qualitatively assessed using two validated instruments: Assessing the Methodological Quality of Systematic Reviews (AMSTAR2) and Risk Of Bias In Systematic reviews (ROBIS). The characteristics and findings of SRs are also reported. A total of 23 SRs formed the basis of this study. Of the 23, six included randomized controlled trials (RCTs) only. Overall, the AMSTAR2 assessment revealed three studies with high and six studies with low methodological quality, and all the other SRs were judged as having critically low methodological quality. ROBIS revealed only one Cochrane review with a low risk of bias and the others with a high risk of bias. In particular, the assessment of non-randomized studies (NRSIs), appropriateness of ROB assessment, and meta-analysis did not satisfy the criteria in AMSTAR2 assessment. Furthermore, there were a few SRs that interpreted and discussed the results of risk of bias (ROB) and heterogeneity assessment, together with the impact of treatment. Due to the lack of head-to-head comparisons conducted in RCTs, review authors need to use other sources of evidence, such as clinical control trials (CCTs), cohort studies (CS), clinical research (CR), and animal studies. The end result is the presentation of low-quality evidence, with high ROB. Several SRs conducted network meta-analysis as an alternative to head-to-head conventional meta-analysis of RCTs. We suggest that the best methods to generate, access, and assess evidence in situations where RCT evidence is lacking should be discussed on an urgent basis.

Alzheimer's disease (AD) is a prevalent neurodegenerative condition that significantly impacts both individuals and society. This study aims to evaluate the effectiveness of repetitive transcranial magnetic stimulation (rTMS) as a treatment for AD by summarizing the evidence from systematic reviews (SRs) and meta-analyses (MAs). SRs/MAs of rTMS for AD were collected by searching Embase, Web of Science, Cochrane Library, PubMed, CNKI, VIP, Sino-Med, and Wanfang databases. The search was conducted from database creation to January 23, 2024. Methodological quality, reporting quality and risk of bias were assessed using the Assessing Methodological Quality of Systematic Reviews 2 (AMSTAR-2), Risk of Bias in Systematic Reviews (ROBIS) tool and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). In addition, the quality of evidence for outcome measures was assessed using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE). Eight SRs/MAs included in this study met the inclusion criteria. Based on the AMSTAR-2, 4 of the SRs/MA were classified as low quality, while the remaining 4 were deemed to be of very low quality. The PRISMA analysis revealed that out of the 27 items reporting, 16 achieved full reporting (100%). However, there were still some deficiencies in reporting, particularly related to protocol and registration, search strategy, risk of bias, and additional analysis. The ROBIS tool indicated that only 3 SRs/MAs had a low risk of bias. The GRADE assessment indicated that 6 outcomes were of moderate quality (18.75%), 16 were of low quality (50%), and 10 were classified as very low quality (31.25%). Based on the evidence collected, rTMS appears to be effective in improving cognitive function in AD patients, although the methodological quality of the SRs/MAs reduces the reliability of the conclusions and the overall quality is low. However, based on the available results, we still support the value of rTMS as an intervention to improve cognitive function in AD. In future studies, it is necessary to confirm the efficacy of rTMS in AD patients and provide more reliable and scientific data to contribute to evidence-based medicine.

Spin Is Prevalent in the Majority of Abstracts of Patello-femoral Arthroplasty Studies.

This study aimed to evaluate and compare the performance of two methodological instruments to appraise systematic reviews and to identify potential disagreements of systematic review authors regarding risk of bias (RoB) evaluation of randomized controlled trials (RCTs) included in systematic reviews on peri-implant diseases. We searched Medline, Web of Science, Cochrane Library, PubMed Central, and Google Scholar for systematic reviews on peri-implant diseases published before July 11, 2017. Two authors independently evaluated the RoB and methodological quality of the systematic reviews by applying the Risk of Bias in Systematic Reviews (ROBIS) tool and Assessing the Methodological Quality of Systematic Reviews (AMSTAR) checklist, respectively. We assessed the RoB scores of the same RCTs published in different systematic reviews. Of the 32 systematic reviews identified, 23 reviews addressed the clinical topic of peri-implantitis. A high RoB was detected for most systematic reviews (n=25) using ROBIS, whilst five systematic reviews displayed low methodological quality by AMSTAR. Almost 30% of the RoB comparisons (for the same RCTs) had different RoB ratings across systematic reviews. The ROBIS tool appears to provide more conservative results than AMSTAR checklist. Considerable disagreement was found among systematic review authors rating the same RCT included in different systematic reviews.

BackgroundThe overall confidence in the results of systematic reviews including animal models can be heterogeneous. We assessed the methodological quality of systematic reviews including animal models in dentistry as well as the overall confidence in the results of those systematic reviews.Material & methodsPubMed, Web of Science and Scopus were searched for systematic reviews including animal studies in dentistry published later than January 2010 until 18th of July 2022. Overall confidence in the results was assessed using a modified version of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) checklist. Checklist items were rated as yes, partial yes, no and not applicable. Linear regression analysis was used to investigate associations between systematic review characteristics and the overall adherence to the AMSTAR-2 checklist. The overall confidence in the results was calculated based on the number of critical and non-critical weaknesses presented in the AMSTAR-2 items and rated as high, moderate, low and critical low.ResultsOf initially 951 retrieved systematic reviews, 190 were included in the study. The overall confidence in the results was low in 43 (22.6%) and critically low in 133 (70.0%) systematic reviews. While some AMSTAR-2 items were regularly reported (e.g. conflict of interest, selection in duplicate), others were not (e.g. funding: n = 1; 0.5%). Multivariable linear regression analysis showed that the adherence scores of AMSTAR-2 was significantly associated with publication year, journal impact factor (IF), topic, and the use of tools to assess risk of bias (RoB) of the systematic reviews.ConclusionAlthough the methodological quality of dental systematic reviews of animal models improved over the years, it is still suboptimal. The overall confidence in the results was mostly low or critically low. Systematic reviews, which were published later, published in a journal with a higher IF, focused on non-surgery topics, and used at least one tool to assess RoB correlated with greater adherence to the AMSTAR-2 guidelines.

BackgroundAn increasing number of systematic reviews assessed the safety of surgical interventions over time. How well these systematic reviews were designed and conducted determines the reliability of evidence. In this study, we aimed to assess the methodological quality of systematic reviews on the safety of surgical interventions.MethodsWe searched PubMed for systematic reviews of surgical interventions with safety as the exclusive outcome from 1st-Jan, 2015 to 1st-Jan, 2020. The methodological quality of eligible systematic reviews was evaluated according to the AMSTAR 2.0 instrument. The primary outcomes were the number of methodological weaknesses and the global methodological quality. The proportion of each methodological weakness among eligible systematic reviews was compared by three pre-defined stratification variables. The absolute difference of the proportion (PD) was used as the effect estimator, with the two-tailed z-test for the significance.ResultsWe identified 127 systematic reviews from 18,636 records. None (n = 0, 0.00%) of them could be rated as “high” in terms of the global methodological quality; in contrast, they were either rated as “low” (n = 18, 14.17%) or as “critically low” (n = 109, 85.83%). The median number of methodological weaknesses of these systematic reviews was 8 (interquartile range, IQR: 6 to 9), in which 4 (IQR: 2 to 4) were critical weaknesses. Systematic reviews that used any reporting guideline (e.g., domain 13, PD = -0.22, 95% CI: − 0.39, − 0.06; p = 0.01) and developed a protocol in advance (e.g., domain 6, PD = -0.20, 95% CI: − 0.39, − 0.01; p = 0.04) were less likely to have methodological weakness in some domains but not for the rest (e.g., domain 8, PD = 0.04, 95% CI: − 0.14, 0.21; p = 0.68; with protocol vs. without).ConclusionsThe methodological quality of current systematic reviews of adverse events with surgical interventions was poor. Further efforts, for example, encouraging researchers to develop a protocol in advance, are needed to enhance the methodological quality of these systematic reviews.

Background:To evaluate the methodological quality of systematic reviews (SRs) or meta-analysis of trastuzumab-based therapy for breast cancer.Methods:We searched the PubMed, EMBASE, Web of science, Cochrane library, international prospective register of systematic reviews, Chinese BioMedical Literature Database, Wan Fang, China National Knowledge Infrastructure and VIP database for SRs or meta-analysis. The methodological quality of included literatures was appraised by risk of bias in systematic review (ROBIS) tool.Results:Twenty three eligible systematic reviews or meta-analysis were included. Only 2 systematic reviews provided protocol. The most frequently searched databases were PubMed, MEDLINE, EMBASE, and the Cochrane. The two-reviewers model described in the screening for eligible original articles, data extraction, and methodological quality evaluation had 30%, 61%, and 26%, respectively. In methodological quality assessment, 52% SRs or meta-analysis used the Jadad scoring or Cochrane reviewer’ handbook. Research question were well matched to all SRs or meta-analysis in phase 1 and 35% of them evaluated “high” risk bias in study eligibility criteria. The “high” risk of bias in all non-Cochrane SRs or meta-analyses, which involve methods used to identify and/or select studies. And more than half SRs or meta-analysis had a high risk of bias in data collection and study appraisal. More than two-third of SRs or meta-analysis were accomplished with high risk of bias in the synthesis and findings.Conclusions:The study indicated poor methodological and reporting quality of SRs/meta-analysis assessing trastuzumab-based therapy for breast cancer. Registration or publishing the protocol and the reporting followed the PRISMA checklist are recommended in future research.

Assessment of the Methodological Quality of Systematic Reviews Published in the Urological Literature From 1998 to 2008

IntroductionAnogenital warts (AGWs) are among the most common STDs. Many therapy options are available but are not codified. Systematic reviews (SRs) and meta-analyses (MAs) are helpful to elaborate recommendations on...

BackgroundArticle summaries’ information and structure may influence researchers/clinicians’ decisions to conduct deeper full-text analyses. Specifically, abstracts of systematic reviews (SRs) and meta-analyses (MA) should provide structured summaries for quick assessment. This study explored a method for determining the methodological quality and bias risk of full-text reviews using abstract information alone.MethodsSystematic literature searches for SRs and/or MA about psoriasis were undertaken on MEDLINE, EMBASE, and Cochrane database. For each review, quality, abstract-reporting completeness, full-text methodological quality, and bias risk were evaluated using Preferred Reporting Items for Systematic Reviews and Meta-analyses for abstracts (PRISMA-A), Assessing the Methodological Quality of Systematic Reviews (AMSTAR), and ROBIS tools, respectively. Article-, author-, and journal-derived metadata were systematically extracted from eligible studies using a piloted template, and explanatory variables concerning abstract-reporting quality were assessed using univariate and multivariate-regression models. Two classification models concerning SRs’ methodological quality and bias risk were developed based on per-item and total PRISMA-A scores and decision-tree algorithms. This work was supported, in part, by project ICI1400136 (JR). No funding was received from any pharmaceutical company.ResultsThis study analysed 139 SRs on psoriasis interventions. On average, they featured 56.7% of PRISMA-A items. The mean total PRISMA-A score was significantly higher for high-methodological-quality SRs than for moderate- and low-methodological-quality reviews. SRs with low-bias risk showed higher total PRISMA-A values than reviews with high-bias risk. In the final model, only ’authors per review > 6’ (OR: 1.098; 95%CI: 1.012-1.194), ’academic source of funding’ (OR: 3.630; 95%CI: 1.788-7.542), and ’PRISMA-endorsed journal’ (OR: 4.370; 95%CI: 1.785-10.98) predicted PRISMA-A variability. Reviews with a total PRISMA-A score < 6, lacking identification as SR or MA in the title, and lacking explanation concerning bias risk assessment methods were classified as low-methodological quality. s with a total PRISMA-A score ≥ 9, including main outcomes results and explanation bias risk assessment method were classified as having low-bias risk.ConclusionsThe methodological quality and bias risk of SRs may be determined by abstract’s quality and completeness analyses. Our proposal aimed to facilitate synthesis of evidence evaluation by clinical professionals lacking methodological skills. External validation is necessary.

Attention deficit hyperactivity disorder (ADHD) is one of the common neurodevelopmental disorders and is widely prevalent worldwide. The primary symptoms of ADHD include inattention, impulsivity, and hyperactivity, which significantly impact the cognitive, behavioral, and emotional dimensions of individuals. These disorders often continue throughout adulthood and, along with associated complications, affect various domains such as personal health, academic achievement, and social interactions. The pathogenesis and contributing causes of ADHD remain unclear at present. Therefore, this study aims to perform an umbrella review of systematic reviews and meta-analyses (SRMAs) to systematically assess the quality of methodologies, potential biases, and validity of all epidemiological evidence related to risk factors for ADHD while offering a comprehensive summary of the evidence regarding these risk factors. This study follows the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) and Cochrane Handbook. We will systematically search 6 databases, including The Cochrane Library Central, PubMed, Embase, Web of Science, CINAHL, and Scopus from the initial period up until 2024 (last update). We will assess the quality of the included SRMAs using the tool to assess risk of bias in systematic reviews (ROBIS), the methodological quality of systematic reviews (AMSTAR)-2, PRISMA-2020, and the grade of recommendations assessment, development and evaluation (GRADE). Two authors will use the ecological models of health behavior to classify the causes and risk factors of ADHD. Finally, we will provide descriptive and comprehensive recommendations for clinical practice and future research. PROSPERO (CRD42024597126).