Methodological Quality Of Meta-analyses Research Articles

Diet and gastric cancer risk: an umbrella review of systematic reviews and meta-analyses of prospective cohort studies.

Several systematic reviews and meta-analyses evaluated the associations between dietary factors and the incidence of gastric cancer (GC). To evaluate the strength and validity of existing evidence, we conducted an umbrella review of published systematic reviews and meta-analyses that investigated the association between diets and GC incidence. We searched the PubMed, Embase, and Cochrane databases for systematic reviews and meta-analyses of prospective cohort studies investigating the association between dietary factors and GC risk. For each association, we recalculated the adjusted summary estimates with their 95% confidence interval (CI) and 95% prediction interval (PI) using a random-effects model. We used the I2 statistic and Egger's test to assess heterogeneity and small-study effects, respectively. We also assessed the methodological quality of each study and the quality of evidence. Finally, we identified 16 meta-analyses that described 57 associations in this umbrella review. Of the 57 associations, eight were statistically significant using random-effects, thirteen demonstrated substantial heterogeneity between studies (I2 > 50%), and three found small-study effects. The methodological quality of meta-analyses was classified as critically low for two (13%), low for thirteen (81%), and only one (6%) was rated as high confidence. Quality of evidence was rated high for a positive association for GC incidence with a higher intake of total alcohol (RR = 1.19, 95% CI 1.06-1.34) and moderate-quality evidence to support that increased processed meat consumption can increase GC incidence. Three associations (total fruit, vitamin E, and carotenoids) were determined to be supported by low-quality evidence, and two (pickled vegetables/foods and citrus fruit) were supported by very low-quality. Our findings support the dietary recommendations for preventative GC, emphasizing lower intake of alcohol and foods preserved by salting. New evidence suggests a possible role for total fruit, citrus fruit, carotenoids, and vitamin E. More research is needed on diets with lower quality evidence. CRD42021255115.

A study of meta-analyses reporting quality in the large and expanding literature of educational technology

As the empirical literature in educational technology continues to grow, meta-analyses are increasingly being used to synthesise research to inform practice. However, not all meta-analyses are equal. To examine their evolution over the past 30 years, this study systematically analysed the quality of 52 meta-analyses (1988–2017) on educational technology. Methodological and reporting quality is defined here as the completeness of the descriptive and methodological reporting features of meta-analyses. The study employed the Meta-Analysis Methodological Reporting Quality Guide (MMRQG), an instrument designed to assess 22 areas of reporting quality in meta-analyses. Overall, MMRQG scores were negatively related to average effect size (i.e., the higher the quality, the lower the effect size). Owing to the presence of poor-quality syntheses, the contribution of educational technologies to learning has been overestimated, potentially misleading researchers and practitioners. Nine MMRQG items discriminated between higher and lower average effect sizes. A publication date analysis revealed that older reviews (1988–2009) scored significantly lower on the MMRQG than more recent reviews (2010–2017). Although the increase in quality bodes well for the educational technology literature, many recent meta-analyses still show only moderate levels of quality. Identifying and using only best evidence-based research is thus imperative to avoid bias.
 
 Implications for practice or policy:
 
 Educational technology practitioners should make use of meta-analytical findings that systematically synthesise primary research.
 Academics, policymakers and practitioners should consider the methodological quality of meta-analyses as they vary in reliability.
 Academics, policymakers and practitioners could avoid misleading bias in research evidence by using the MMRQG to evaluate the quality of meta-analyses.
 
 Meta-analyses with lower MMRQG scores should be considered with caution as they seem to overestimate the effect of educational technology on learning.

Effectiveness of probiotics in irritable bowel syndrome: methodological quality of meta-analyses and systematic reviews

Abstract Objective This study assessed the methodological quality of systematic reviews/meta-analysis of the effectiveness of probiotics against irritable bowel syndrome (IBS) using the accepted methodological quality assessment scale AMSTAR and explored the factors that influenced the quality of methodology. It was designed to provide a reference for future research and systematic reviews/metaanalysis. Methods The methodological quality of existing systematic reviews/meta-analysis was evaluated using the AMSTAR scale. Influencing factors of methodological quality were statistically analyzed using RevMan 5.3 software. The included systematic reviews/metaanalysis must include the following characteristics: (1) methods using systematic evaluation/meta-analysis, (2) probiotic intervention, and (3) language limitation to Chinese and English. Results The AMSTAR score was 5–9 (7.42 ± 1.22), and the quality is above average. The factors affecting the methodological quality are the number of authors and whether they cooperate with the institution. Conclusions Studies have shown that current systematic reviews/meta-analysis of the effectiveness of probiotics on IBS does not fully comply with methodological quality standards, and therefore the methodological quality of research in this area needs to be strengthened. To better clarify how probiotics affect IBS, future systematic reviews and meta-analyses should not only follow methodological quality standards but also include more effective outcome measures, and they should focus more on the discussion of research results. We look forward to the development of higher-quality randomized controlled trials in the future.

A systematic review of meta-analyses in orthopaedic surgery between 2000 and 2016.

The aims of this systematic review were to describe the quantity and methodological quality of meta-analyses in orthopaedic surgery published during the last 17 years. MEDLINE, EMBASE, and PubMed, between 1 January 2000 and 31 December 2016, were searched for meta-analyses in orthopaedic surgery dealing with at least one surgical intervention. Meta-analyses were included if the interventions involved a human muscle, ligament, bone or joint. A total of 392 meta-analyses met eligibility criteria, for which the mean AMSTAR quality score was 7.1/11. There was a positive correlation between the year of publication and the quality of the meta-analysis (r = 0.238, p < 0.001). Between 2000 and 2011, the mean AMSTAR score corresponded to that of a medium quality review. However, between 2012 and 2016, the mean scores have been consistently equivalent to those of a high-quality review. The number of meta-analyses published increased 10-fold between 2005 and 2014. The quantity and quality of meta-analyses in orthopaedic surgery which have been published has increased, reaching a plateau in 2012. Methodological flaws remain to be addressed in future meta-analyses in order to continue increasing the quality of the orthopaedic literature. Cite this article: Bone Joint J 2018;100-B:1270-4.

Methodological Quality Assessment of Meta-Analyses of Hyperthyroidism Treatment.

Hyperthyroidism is a common condition that is associated with increased morbidity and mortality. A number of meta-analyses (MAs) have assessed the therapeutic measures for hyperthyroidism, including antithyroid drugs, surgery, and radioiodine, however, the methodological quality has not been evaluated. This study evaluated the methodological quality and summarized the evidence obtained from MAs of hyperthyroidism treatments for radioiodine, antithyroid drugs, and surgery. We searched the PubMed, EMBASE, Cochrane Library, Web of Science, and Chinese Biomedical Literature Database databases. Two investigators independently assessed the meta-analyses titles and abstracts for inclusion. Methodological quality was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. A total of 26 MAs fulfilled the inclusion criteria. Based on the AMSTAR scores, the average methodological quality was 8.31, with large variability ranging from 4 to 11. The methodological quality of English meta-analyses was better than that of Chinese meta-analyses. Cochrane reviews had better methodological quality than non-Cochrane reviews due to better study selection and data extraction, the inclusion of unpublished studies, and better reporting of study characteristics. The authors did not report conflicts of interest in 53.8% meta-analyses, and 19.2% did not report the harmful effects of treatment. Publication bias was not assessed in 38.5% of meta-analyses, and 19.2% did not report the follow-up time. Large-scale assessment of methodological quality of meta-analyses of hyperthyroidism treatment highlighted methodological strengths and weaknesses. Consideration of scientific quality when formulating conclusions should be made explicit. Future meta-analyses should improve on reporting conflict of interest.

Reporting and methodological quality of meta-analyses in urological literature.

PurposeTo assess the overall quality of published urological meta-analyses and identify predictive factors for high quality.Materials and MethodsWe systematically searched PubMed to identify meta-analyses published from January 1st, 2011 to December 31st, 2015 in 10 predetermined major paper-based urology journals. The characteristics of the included meta-analyses were collected, and their reporting and methodological qualities were assessed by the PRISMA checklist (27 items) and AMSTAR tool (11 items), respectively. Descriptive statistics were used for individual items as a measure of overall compliance, and PRISMA and AMSTAR scores were calculated as the sum of adequately reported domains. Logistic regression was used to identify predictive factors for high qualities.ResultsA total of 183 meta-analyses were included. The mean PRISMA and AMSTAR scores were 22.74 ± 2.04 and 7.57 ± 1.41, respectively. PRISMA item 5, protocol and registration, items 15 and 22, risk of bias across studies, items 16 and 23, additional analysis had less than 50% adherence. AMSTAR item 1, “a priori” design, item 5, list of studies and item 10, publication bias had less than 50% adherence. Logistic regression analyses showed that funding support and “a priori” design were associated with superior reporting quality, following PRISMA guideline and “a priori” design were associated with superior methodological quality.ConclusionsReporting and methodological qualities of recently published meta-analyses in major paper-based urology journals are generally good. Further improvement could potentially be achieved by strictly adhering to PRISMA guideline and having “a priori” protocol.

Exercise and sleep: a systematic review of previous meta-analyses.

Conduct a systematic review of previous meta-analyses on exercise and sleep outcomes in adults and a meta-analysis of studies nested within these meta-analyses. Meta-analyses of randomized controlled exercise interventions were included by searching nine electronic databases and cross-referencing. Dual-selection and data abstraction were conducted. Methodological quality of meta-analyses was assessed using AMSTAR and quality of evidence using GRADE. Random-effects models were used to pool results from the individual studies included in each meta-analysis. Three meta-analyses representing 950 adults were included. Methodological quality ranged from 36% to 64% while quality of evidence was very low to low. Statistically significant improvements (P ≤ 0.05) were observed for the apnea-hypopnea index (AHI), overall sleep quality, global score, subjective sleep, and sleep latency. The number-needed-to-treat (NNT) and percentile improvements ranged from 4 to 7 and from 18.1 to 26.5, respectively. When overall sleep quality results from individual studies nested within different meta-analyses were pooled, statistically significant standardized mean difference (SMD) improvements were observed (-0.50, 95% CI -0.72 to -0.28). The NNT and percentile improvement were 7 and 19, respectively. Exercise improves selected sleep outcomes in adults. To increase public health reach, a large, well-designed, and more inclusive meta-analysis is needed.

Epidemiological characteristics and methodological quality of meta-analyses on diabetes mellitus treatment: a systematic review.

Well-conducted meta-analyses (MAs) are considered as one of the best sources of clinical evidence for treatment decision. MA with methodological flaws may introduce bias and mislead evidence users. The aim of this study is to investigate the characteristics and methodological quality of MAs on diabetes mellitus (DM) treatments. Systematic review. Cochrane Database of Systematic Review and Database of Abstract of Reviews of Effects were searched for relevant MAs. Assessing methodological quality of systematic reviews (AMSTAR) tool was used to evaluate the methodological quality of included MAs. Logistic regression analysis was used to identify association between characteristics of MA and AMSTAR results. A total of 252 MAs including 4999 primary studies and 13,577,025 patients were included. Over half of the MAs (65.1%) only included type 2 DM patients and 160 MAs (63.5%) focused on pharmacological treatments. About 89.7% MAs performed comprehensive literature search and 89.3% provided characteristics of included studies. Included MAs generally had poor performance on the remaining AMSTAR items, especially in assessing publication bias (39.3%), providing lists of studies (19.0%) and declaring source of support comprehensively (7.5%). Only 62.7% MAs mentioned about harm of interventions. MAs with corresponding author from Asia performed less well in providing MA protocol than those from Europe. Methodological quality of MA on DM treatments was unsatisfactory. There is considerable room for improvement, especially in assessing publication bias, providing lists of studies and declaring source of support comprehensively. Also, there is an urgent need for MA authors to report treatment harm comprehensively.

Methodologic quality of meta-analyses and systematic reviews on the Mediterranean diet and cardiovascular disease outcomes: a review

AbstractBackground: Several systematic reviews/meta-analyses published within the past 10 y have examined the associations of Mediterranean-style diets (MedSDs) on cardiovascular disease (CVD) risk. However, these reviews have not been evaluated for satisfying contemporary methodologic quality standards.Objective: This study evaluated the quality of recent systematic reviews/meta-analyses on MedSD and CVD risk outcomes by using an established methodologic quality scale. The relation between review quality and impact per publication value of the journal in which the article had been published was also evaluated.Design: To assess compliance with current standards, we applied a modified version of the Assessment of Multiple Systematic Reviews (AMSTARMedSD) quality scale to systematic reviews/meta-analyses retrieved from electronic databases that had met our selection criteria: 1) used systematic or meta-analytic procedures to review the literature, 2) examined MedSD trials, and 3) had MedSD interventions independently or combined with other interventions.Results: Reviews completely satisfied from 8% to 75% of the AMSTARMedSD items (mean ± SD: 31.2% ± 19.4%), with those published in higher-impact journals having greater quality scores. At a minimum, 60% of the 24 reviews did not disclose full search details or apply appropriate statistical methods to combine study findings. Only 5 of the reviews included participant or study characteristics in their analyses, and none evaluated MedSD diet characteristics.Conclusions: These data suggest that current meta-analyses/systematic reviews evaluating the effect of MedSD on CVD risk do not fully comply with contemporary methodologic quality standards. As a result, there are more research questions to answer to enhance our understanding of how MedSD affects CVD risk or how these effects may be modified by the participant or MedSD characteristics. To clarify the associations between MedSD and CVD risk, future meta-analyses and systematic reviews should not only follow methodologic quality standards but also include more statistical modeling results when data allow.

Mega-review of meta-analyses investigating the short-term efficacy of pharmacologic augmentation strategies of antipsychotics in patients with schizophrenia

Antipsychotics are the cornerstone of treatment for schizophrenia, but they have limited effectiveness, as most patients require subsequent strategies at some point of their treatment. Despite being widely used, the efficacy of pharmacologic augmentation of antipsychotics is controversial and no combination treatment has been approved for schizophrenia. We conducted a systematic review in PubMed and PsycInfo on June 1st 2015 and a random effects meta-analysis of meta-analyses of short-term, placebo-controlled studies of pharmacological augmentation strategies of antipsychotics in schizophrenia. Methodological quality of meta-analyses was measured using the AMSTAR, plus 6 additional items developed to rate the content quality of the meta-analyzed trials. Out of 3062 publications, we identified 36 eligible augmenting strategies. For total symptom reduction, 25 strategies augmenting antipsychotics and 5 strategies augmenting clozapine were eligible and examined. Eleven strategies were more efficacious than placebo, none of them augmenting clozapine. Significant effect sizes ranged between SMD −1.03 and −0.23. Efficacy was not correlated with the quality of the meta-analyses. Only the meta-analysis for NSAIDs augmentation had a score greater than half of the possible points for content quality. Only antipsychotics, azapirones, antidepressants and lithium were less discontinued than placebo. Serotonin-3-receptor antagonists, lamotrigine, mirtazapine/mianserine, minocycline and estrogens had large effect sizes augmenting antipsychotics. However the quality of the content of most meta-analyses was low. The NSAIDs augmentation meta-analysis had the best content quality, yet with a low effect size for efficacy. The evidence for short-term augmentation strategies of antipsychotics in schizophrenia is inconclusive, due to the limited quality of the available trials.Disclosure of interestThe authors have not supplied their declaration of competing interest.

Epidemiology, quality and reporting characteristics of meta-analyses of observational studies published in Chinese journals

ObjectiveThe aim of this study was to examine the epidemiological and reporting characteristics as well as the methodological quality of meta-analyses (MAs) of observational studies published in Chinese journals.Methods5 Chinese...

Methodological quality of meta-analyses on treatments for chronic obstructive pulmonary disease: a cross-sectional study using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool.

Background:Meta-analysis (MA) of randomised trials is considered to be one of the best approaches for summarising high-quality evidence on the efficacy and safety of treatments. However, methodological flaws in MAs can reduce the validity of conclusions, subsequently impairing the quality of decision making.Aims:To assess the methodological quality of MAs on COPD treatments.Methods:A cross-sectional study on MAs of COPD trials. MAs published during 2000–2013 were sampled from the Cochrane Database of Systematic Reviews and Database of s of Reviews of Effect. Methodological quality was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool.Results:Seventy-nine MAs were sampled. Only 18% considered the scientific quality of primary studies when formulating conclusions and 49% used appropriate meta-analytic methods to combine findings. The problems were particularly acute among MAs on pharmacological treatments. In 48% of MAs the authors did not report conflict of interest. Fifty-eight percent reported harmful effects of treatment. Publication bias was not assessed in 65% of MAs, and only 10% had searched non-English databases.Conclusions:The methodological quality of the included MAs was disappointing. Consideration of scientific quality when formulating conclusions should be made explicit. Future MAs should improve on reporting conflict of interest and harm, assessment of publication bias, prevention of language bias and use of appropriate meta-analytic methods.

A Methodological Quality Assessment of Meta-Analyses of Rehabilitation Therapy Areas in Korea

The purpose of this literature paper was to assess the methodological quality of meta-analyses on rehabilitation therapy published in Korea by using AMSTAR(A Measurement Tool to Assess the Methodological Quality or Systematic Review). Nine meta-analyses were included in this study. The mean score was 6.56, and all meta-analyses were rated as moderate quality. For the evidence-based practice of rehabilitation therapy areas, high quality meta-analyses completed with the methodological guidelines should be activated. For this purpose, the accumulation of individual studies should be preceded.

Methodological Quality of Meta-Analyses on the Elderly in Korea

Methodological Quality of Meta-Analyses on the Elderly in Korea

Methodological quality of meta-analyses on the blood pressure response to exercise

Numerous meta-analyses have been conducted to summarize the growing numbers of trials addressing the effects of exercise on blood pressure (BP), yet it is unclear how well they have satisfied contemporary methodological standards. We applied an augmented version of the Assessment of Multiple Systematic Reviews (AMSTARExBP) scale to 33 meta-analyses retrieved from searches of electronic databases. Qualifying reports used meta-analytic procedures; examined controlled exercise training trials; had BP as a primary outcome; and had exercise or physical activity interventions independently or combined with other lifestyle interventions. AMSTARExBP scores averaged near the middle of the scale (Mean = 56.0% ± 21.4% of total items possible); co-authored and more recent meta-analyses had higher quality scores. Common deficits were failures to disclose full search details (30% did), gauge the quality of included trials (48% did), use duplicate study selection and data extraction (55% did), or incorporate study quality in formulating results (35% did). Nearly all (91%) meta-analyses observed that exercise significantly lowered BP; fewer (58%) found that such effects depended on exercise or patient characteristics but these patterns often conflicted. Meta-analyses are often pillars of clinical recommendations and guidelines, yet only 58% addressed the clinical translations of their findings. In sum, meta-analyses have contributed less than ideally to our understanding of how exercise may impact BP, or how these BP effects may be moderated by patient or exercise characteristics. Future meta-analyses that better satisfy contemporary standards offer considerable promise to understand how and for whom exercise impacts BP.

A systematic review evaluating the potential for bias and the methodological quality of meta-analyses in vaccinology

A systematic review was undertaken to produce an annotated bibliography of meta-analyses in vaccinology and to evaluate their methodological quality. Based on our evaluation using the Oxman and Guyatt index, the methodological quality of the 121 meta-analyses included in this study is not satisfactory. The most frequent limitations include non-comprehensive bibliographic research; bias in the selection of the studies; lack of quality assessment of individual studies; absence of evaluation of heterogeneity among studies and publication bias. The methodological quality significantly increases with the year of publication and with declared financial support, without differences between profit and non-profit support. Meta-analyses with a higher Oxman and Guyatt quality score are more likely to include only randomized trials and to explore appropriately potential sources of heterogeneity. Most of the methodological deficiencies of meta-analyses in vaccinology could be corrected easily, and meta-analysts should improve the methodological quality of their work to maintain their impact on policy decisions.