Meta-epidemiological Study Research Articles

Incomplete reporting of patient-reported outcomes in multiple sclerosis: A meta-epidemiological study of randomized controlled trials

BackgroundMultiple Sclerosis significantly affects quality of life, which is often measured by patient-reported outcomes. The incorporation of patient-reported outcomes within clinical trials supplements the efficacy of outcomes in order to provide a patient's perspective for clinicians. Our objective was to evaluate current literature for completeness of reporting of PROs in randomized controlled trials (RCTs) for the management of MS. MethodsWe used MEDLINE, Embase, and Cochrane Central Register of Controlled Trials to search for RCT publications investigating the management of MS. After duplicate screening via Rayyan, RCTs fitting our inclusion criteria were abstracted employing the Consolidated Standards of Reporting Trials - Patient-Reported Outcome (CONSORT-PRO) adaptation and the Cochrane Collaboration Risk of Bias (RoB) 2.0 tool. Mean percent completion of an adaptation of CONSORT-PRO was calculated to address completeness of reporting. In addition, bivariate regression models were used to evaluate relationships between trial characteristics and completeness of reporting. ResultsOur search returned 3,966 results and 92 RCTs were included for data abstraction and analysis. We found an overall completion of 48.68% (SD=19.03). Sixty-five (of 92; 70.65%) of the RCTs were evaluated as having ‘high’ RoB. There were significant associations between completeness of reporting and the following: mention of CONSORT within published RCTs (t=2.55, p=.013), length of PRO follow-up (t=2.9, p=.005; t=2.14, p=.035), and sample size (t=3.12, p=.002). No other significant associations were found. ConclusionOur study found incomplete adherence to the CONSORT-PRO adaptation among RCTs pertaining to MS. Of the most underreported items, the failure to report a hypothesis and define an approach to missing data threaten the validity of the evidence acquired from RCTs. Furthermore, PROs provide an opportunity to supplement trial outcomes with the patient's perspective. Thus, trialists of future RCTs may improve PRO reporting with increased adherence to the CONSORT-PRO adaptation.

Methodological Quality of Physical Therapy-Related Trials Published in Open Access and Subscription Journal: A Cross-sectional Meta-Epidemiological Study.

We aimed to compare the methodological quality of physical therapy-related trials published in open access with that of trials published in subscription-based journals, adjusting for subdiscipline, intervention type, endorsement of the Consolidated Standards of Reporting Trials, impact factor, and publication language. In this meta-epidemiological study, we searched the Physiotherapy Evidence Database on May 8, 2021, to include any physical therapy-related trials published from January 1, 2020. We extracted variables such as Consolidated Standards of Reporting Trials endorsement, the Physiotherapy Evidence Database score, and publication type. We compared the Physiotherapy Evidence Database score between the publication types using a multivariable generalized estimating equation by adjusting for covariates. A total of 2743 trials were included, with a mean total Physiotherapy Evidence Database score (standard deviation) of 5.8 (±1.5). Trials from open access journals had a lower total Physiotherapy Evidence Database score than those from subscription-based journals (5.5 ± 1.5 vs. 5.9 ± 1.5, mean difference = -0.4; 95% confidence interval = 0.3-0.5). Generalized estimating equation revealed that open access publication was significantly associated with the total Physiotherapy Evidence Database score (mean difference = -0.42; P < 0.001). In the recent physical therapy-related trials, open access publications demonstrated lower methodological quality than subscription-based publications, although with a small difference.

Bias in Hand Surgical Randomized Controlled Trials: Systematic Review and Meta-Epidemiological Study

Inappropriately reported or conducted studies may decrease the quality of care due to under- or overestimation of the benefits or harms of interventions. Our aim was to evaluate how often hand surgical randomized controlled trials (RCTs) use and report adequate methods to ensure internal validity, and whether inadequate reporting or methods are associated with the magnitude of treatment effect estimates. Data Sources were the Cochrane Central Register of Controlled Trials, MEDLINE, and Embase databases until November 2020. We included published RCTs investigating the effects of any surgical intervention in the hand and wrist region. We assessed internal validity using the Cochrane Risk of Bias (RoB) tool for 6 domains: selection, performance, detection, attrition, selective reporting, and "other" bias. We extracted the primary outcome and calculated the effect size for each study. We used mixed-effect meta-regression to assess whether the RoB modified the magnitude of the effects. For 207 assessed trials, the RoB was unclear or high for 72% in selection, 93% in performance, 88% in detection, 25% in attrition, 22% in selective reporting, and 34% in the "other" bias domain. Trials with a high or unclear risk of selection bias yielded 0.28 standardized mean difference (95% confidence interval, 0.02-0.55) larger effect sizes compared to studies with a low risk. Risks of bias for other domains did not modify the intervention effects. The risk for selection bias declined over time: the odds ratio for a high or unclear RoB was 0.90 (95% confidence interval, 0.85-0.95) per additional year of publication CONCLUSIONS: The internal validity and credibility of hand surgical RCTs can be improved by using established methods to achieve true randomization, blinding of the participants and study personnel, publishing the trial protocol and avoiding selective reporting of the outcomes, and reporting the trial as recommended in the Consolidated Standards of Reporting Trials statement. Clinicians should be aware that RCTs that do not use or report proper randomization and allocation concealment may overestimate the treatment effects.

The evaluation of reporting of patient-reported outcomes in MDD: A meta-epidemiological study of clinical trials

Major depressive disorder (MDD) is a multifaceted disease that profoundly affects quality of life. Patient reported outcomes (PROs) are used in randomized controlled trials (RCTs) to better understand patient perspectives on interventions. Therefore, we sought to assess the completeness of reporting PROs in RCTs addressing MDD. We identified RCTs evaluating MDD containing a PRO measure published between 2016 and 2020 from MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials. Inclusion of studies was performed in duplicate. The completion of reporting of RCTs was assessed using the Consolidated Standards of Reporting Trials (CONSORT-PRO) adaptation. Bivariate regression analyses were used to evaluate reporting completeness and trial characteristics. A total of 49 RCTs were included in our analysis, with a mean CONSORT-PRO completion score of 56.7% (SD = 17.3).Our findings show a significant association with completeness of reporting and the following: secondary PRO trials were less completely reported as compared to primary PRO trials (t = −3.19, p = .003); studies with a follow-up period between six months and year were more completely reported as compared to three months or less (6 months to a year, t = 2.34, p = .024); and increased trial sample size was associated with more completeness of reporting (t = 3.17, p = .003). As compared to brain stimulation, the intervention types classified as combination, other, and psychotherapy had greater completeness of reporting (combination, t = 2.35, p = .024; other, t = 3.13, p = .003; psychotherapy, t = 3.41, p = .001). There were no other significant findings. Our study found the completeness of PRO reporting to be inconsistent in RCTs regarding MDD. Moreover, we advocate for the need to establish a core outcome set relevant to the management of adults diagnosed with MDD and facilitate training on the application of PRO data.

Health disparities, COVID-19, and maternal and childbirth outcomes: a meta-epidemiological study of equity reporting in systematic reviews

Background Pregnant women with COVID-19 are at increased risk for adverse maternal and pregnancy outcomes, and birth complications. Given the health outcome disparities among pregnant women of racial and ethnic minorities and the reliance of medical practice on systematic reviews and meta-analyses (SRMAs)—as they are the apical component in the hierarchy of evidence in medical research—the primary objective of the study is to examine the inclusion of the equity reporting in SRMAs focused on pregnancy outcomes and COVID-19 using PROGRESS-Plus equity framework. PROGRESS represents equity measures of Place, Race, Occupation, Gender, Religion, Education, Social capital, and Socio-economic status. Methods We conducted a systematic search of three databases to identify SRMAs related to maternal and pregnancy outcomes related to COVID-19. We extracted whether SRMAs reported or analyzed PROGRESS-Plus components among other study characteristics. Results Nearly 85% of SRMAs did not include any equity items to account for racial or geographic disparities. Reporting of race was absent from 95% of the studies. Place was the most common PROGRESS item and maternal age was the most common PROGRESS-Plus item reported overall. Conclusion When research is performed and reported in a way that fails to address disparities, the downstream repercussions may include medical care in the form of new protocol-driven hospital management, pharmacologic interventions, and other treatment options that mirror this absence in reporting. The absence of adequate reporting widens gaps in health outcomes among at-risk groups, such as pregnant women of racial and ethnic minorities.

Thresholds for interpreting the fragility index derived from sample of randomised controlled trials in cardiology: a meta-epidemiologic study

The fragility index (FI) was proposed as a simplified way to communicate robustness of statistically significant results and their susceptibility to a change of a handful number of events. While...

Meta-analyses frequently include old trials that are associated with a larger intervention effect: a meta-epidemiological study

ObjectiveTo assess whether meta-analyses include older randomized controlled trials (RCTs) and whether intervention effect differ between older and recent RCTs. Study Design and SettingIn this meta-epidemiological study of 295 meta-analyses (2940 RCTs) published in 2017–2018, we evaluated the difference in intervention effects between older (i.e., published before 2000) and recent RCTs. We also compared effects by quarters of publication year within each meta-analysis (from quarter 1 including the 25% oldest trials to quarter 4 including the 25% most recent trials). A ratio of odds ratio (ROR) <1 indicates larger effects in older than recent RCTs. ResultsTrials published before 2000 and before 1990 represented 25% and 10% of all trials, respectively. Intervention effects were significantly larger for old than recent RCTs (ROR = 0.92, 95% confidence interval [CI] 0.85–1.00, I2 = 22%). Compared with the most recent trials (quarter 4), intervention effects were significantly larger for the oldest trials (quarter 1) (ROR = 0.85, 95% CI 0.79–0.92) and for trials in quarter 2 (ROR = 0.89, 95% CI 0.83–0.96) but not for trials in quarter 3 (ROR = 0.98, 95% CI 0.91–1.05). ConclusionsIntervention effects were larger for older than recent RCTs. Meta-analyses including older trials should be interpreted cautiously.

Retracted Randomised Controlled Trials Were Cited and Not Corrected in Systematic Reviews and Clinical Practice Guidelines: A Meta-Epidemiological Study

Retracted Randomised Controlled Trials Were Cited and Not Corrected in Systematic Reviews and Clinical Practice Guidelines: A Meta-Epidemiological Study

Most Healthcare Interventions Tested in Cochrane Reviews are Not Effective According to High Quality Evidence: A Meta-Epidemiological Study

Most Healthcare Interventions Tested in Cochrane Reviews are Not Effective According to High Quality Evidence: A Meta-Epidemiological Study

Opportunities to improve reporting of rapid response in health technology assessment.

Mini health technology assessment (HTA) reports have been used to support policy makers and health systems by providing a timely summary of scientific evidence. The objective of this meta-epidemiologic study was to evaluate the quality of reporting of mini-HTA reports published in Brazil. An electronic search for all mini-HTA reports published between 2014 and March 2019 was conducted in the SISREBRATS and CONITEC databases. The study selection and data extraction were performed by two independent assessors. The following data were extracted: bibliographic data; research question; characteristics of the population, health technologies and outcomes assessed; eligibility criteria; information about searches and study selection; risk of bias assessment; quality of evidence assessment; synthesis of results; and recommendation about the technology evaluated. A descriptive analysis was used to summarize the information retrieved from all the included mini-HTA reports. We included 103 mini-HTA reports, the great majority of which (92.3 percent) focused on the coverage of the technologies in the healthcare system, with more than 60 percent being about drugs. Only five mini-HTA reports (4.8 percent) gave reasons for the choice of outcomes, and fifteen (14.5 percent) discriminated between primary and secondary outcomes. All mini-HTAs reported the databases searched and 99 percent of them reported using Medline. Sixty percent of the mini-HTA reported assessing the risk of bias, and 52 percent reported assessing the quality of evidence. The quality of reporting of the mini-HTA reports performed in Brazil is insufficient and needs to be improved to guarantee transparency and replicability.

Problems with the MetaBLIND study: An examination of data on blinding patients in trials with patient-reported outcomes.

MetaBLIND is the largest meta-epidemiological study on the impact of blinding in randomized trials to date. We examined MetaBLIND data on the impact of blinding patients on patient-reported outcomes. 68 out of 132 included trials tested knowledge recall and had questionable relevance to clinical trials. In 17 out of 18 comparisons, the number of trials in the blinded or nonblinded group was 2 or lower. In several key trials, the blinding status was uncertain. Effect sizes compared in MetaBLIND appear to reflect random differences in study design and setting rather than the impact of blinding trial participants.

Assessing Treatment Success or Failure as an Outcome in Randomised Clinical Trials of COPD Exacerbations. A Meta-Epidemiological Study.

A recently published ERS core outcome set recommends that all trials of COPD exacerbation management should assess the treatment success (or “cure” of the exacerbation), defined as a dichotomous measure of the overall outcome of an exacerbation. This methodological systematic review describes and compares the instruments that were used to assess treatment success or failure in 54 such RCTs, published between 2006–2020. Twenty-three RCTs used composite measures consisting of several undesirable outcomes of an exacerbation, together defining an overall unfavourable outcome, to define treatment failure. Thirty-four RCTs used descriptive instruments that used qualitative or semi-quantitative descriptions to define cure, marked improvement, improvement of the exacerbation, or treatment failure. Treatment success and failure rates among patients receiving guidelines-directed treatments at different settings and timepoints are described and could be used to inform power calculations in future trials. Descriptive instruments appeared more sensitive to treatment effects compared to composite instruments. Further methodological studies are needed to optimise the evaluation of treatment success/failure. In the meantime, based on the findings of this systematic review, the ERS core outcome set recommends that cure should be defined as sufficient improvement of the signs and symptoms of the exacerbation such that no additional systemic treatments are required.

The Impact of Drug Trials With Financial Conflict of Interests on the Meta-analyses: A Meta-epidemiological Study.

To assess the impact of trials with potential financial conflict of interests (FCOIs) on evidence synthesis in meta-analyses (MAs). A total of 96 MAs from the Cochrane Library about drug trials were investigated. The primary outcomes examined the proportion of conclusions that would change with the exclusion of trials with potential FCOIs. If the proportion of changed conclusions was below the non-inferiority margin of 10%, we considered that it was not inferior to include the trials with potential FCOIs in the MAs. Only 54.17% of MAs reported the funding sources of each included trial, and in 21.88% of MAs, the author-industry-related financial ties of each included trial were reported. When trials with FCOIs were excluded, the changed conclusions of effectiveness and major adverse events were 13.16% and 11.11%, respectively, and the I2 decreased by 13.56% and 10.09%, respectively. For serious adverse events, the exclusion of FCOIs trials did not lead to any change in conclusions; however, the I2 decreased by 24.24%. The impact of trials without reported FCOIs was also examined on evidence synthesis, and the results showed that the changed conclusions of effectiveness and major adverse events were 5.26% and 6.25%, respectively, indicating non-inferiority. However, the I2 increased by 13.60% and 12.37%, respectively. In this meta-epidemiological study, we demonstrated that trials with FCOIs may not only influence the final outcome of MAs but may also increase the heterogeneity of results. It is suggested that all MAs fully report the FCOIs involved in evidence-based research and explore the impact of its FCOIs to better provide a more valuable reference for patients, clinicians, and policy-makers.

High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study.

Redundant publication of systematic reviews and meta-analyses (SRs/MAs) on the same topic presents an increasing burden for clinicians. The aim of this study was to describe variabilities in effect size and methodological quality of overlapping surgery-related SRs/MAs and to investigate factors associated with their postpublication citations. PubMed/MEDLINE was searched to identify SRs/MAs of RCTs on thoracoabdominal surgeries published in 2015. Previous SRs/MAs on the same topics published within the preceding 5 years (2011-2015) were identified and 5-year citation counts (through to 2020) were evaluated. Discrepancies in pooled effect sizes and their methodological quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR) among overlapping SRs/MAs were assessed. The SR/MA-level factors associated with 5-year citation counts were explored, using a mixed-effects regression model with a random intercept for surgical topics. A total of 57 surgery-related SRs/MAs (48 topics) published in 2015 were identified, and 146 SRs/MAs had overlapping publications on 29 topics (60.4 per cent of all topics) in the preceding 5 years. There was considerable variability in methodological quality of SRs/MAs and coverage probability for relevant RCTs, resulting in discrepant effect size estimates for the same topic. High quality (AMSTAR score 8-11) was independently associated with higher 5-year citation counts (coefficient = 32.82; 95 per cent c.i. 15.63 to 50.02; P < 0.001). Overlapping SRs/MAs with high variability in results and methodological quality were common in surgery. A high-quality SR/MA score was an independent predictor of more frequent citations. Researchers and journal editors should concentrate their efforts on limiting publications to higher-quality reviews.

Combining meta-epidemiological study datasets on commercial funding of randomised clinical trials: Database, methods, and descriptive results of the COMFIT study.

Randomised trials are often funded by commercial companies and methodological studies support a widely held suspicion that commercial funding may influence trial results and conclusions. However, these studies often have a risk of confounding and reporting bias. The risk of confounding is markedly reduced in meta-epidemiological studies that compare fairly similar trials within meta-analyses, and risk of reporting bias is reduced with access to unpublished data. Therefore, we initiated the COMmercial Funding In Trials (COMFIT) study aimed at investigating the impact of commercial funding on estimated intervention effects in randomised clinical trials based on a consortium of researchers who agreed to share meta-epidemiological study datasets with information on meta-analyses and trials included in meta-epidemiological studies. Here, we describe the COMFIT study, its database, and descriptive results. We included meta-epidemiological studies with published or unpublished data on trial funding source and results or conclusions. We searched five bibliographic databases and other sources. We invited authors of eligible meta-epidemiological studies to join the COMFIT consortium and to share data. The final construction of the COMFIT database involves checking data quality, identifying trial references, harmonising variable categories, and removing non-informative meta-analyses as well as correlated meta-analyses and trial results. We included data from 17 meta-epidemiological studies, covering 728 meta-analyses and 6841 trials. Seven studies (405 meta-analyses, 3272 trials) had not published analyses on the impact of commercial funding, but shared unpublished data on funding source. On this basis, we initiated the construction of a combined database. Once completed, the database will enable comprehensive analyses of the impact of commercial funding on trial results and conclusions with increased statistical power and a markedly reduced risk of confounding and reporting bias.

Patient-important outcomes reported in randomized controlled trials of pharmacologic treatments for COVID-19: a protocol of a META-epidemiological study

BackgroundThe coronavirus disease 19 (covid-19) pandemic has underscored the need to expedite clinical research, which may lead investigators to shift away from measuring patient-important outcomes (PIO), limiting research applicability. We aim to investigate if randomized controlled trials (RCTs) of covid-19 pharmacological therapies include PIOs.MethodsWe will perform a meta-epidemiological study of RCTs that included people at risk for, or with suspected, probable, or confirmed covid-19, examining any pharmacological treatment or blood product aimed at prophylaxis or treatment. We will obtain data from all RCTs identified in a living network metanalysis (NMA). The main data sources are the living WHO covid-19 database up to 1 March 2021 and six additional Chinese databases up to 20 February 2021. Two reviewers independently will review each citation, full-text article, and abstract data. To categorize the outcomes according to their importance to patients, we will adapt a previously defined hierarchy: a) mortality, b) quality of life/ functional status/symptoms, c) morbidity, and d) surrogate outcomes. Outcomes within the category a) and b) will be considered critically important to patients, and outcomes within the category c) will be regarded as important. We will use descriptive statistics to assess the proportion of studies that report each category of outcomes. We will perform univariable and multivariable analysis to explore associations between trial characteristics and the likelihood of reporting PIOs.DiscussionThe findings from this meta-epidemiological study will help health care professionals and researchers understand if the current covid-19 trials are effectively assessing and reporting the outcomes that are important to patients. If a deficiency in capturing PIOs is identified, this information may help inform the development of future RCTs in covid-19.Systematic review registrationsOpen Science Framework registration: osf.io/6xgjz.

The methodological quality was low and conclusions discordant for meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study

ObjectiveTo investigate the association between methodological quality and reported conclusions of meta-analyses comparing operative with non-operative treatments for proximal humerus fractures. Study design and settingCross-sectional meta-epidemiological study. We searched EMBASE, PubMed, The Cochrane Library, and Web of Science for systematic reviews with meta-analyses comparing non-operative with operative treatments for proximal humerus fractures. Methodological quality was assessed using AMSTAR2 and the reported conclusions were scored for three outcome domains (functional outcome, quality of life, and harm) on a scale from 1 to 6. The Mann-Whitney and Kruskal-Wallis tests were used to investigate the association between methodological quality and reported conclusions. ResultsWe included 21 systematic reviews: 19 pairwise meta-analyses and 2 network meta-analyses, although there are only 8 published randomized controlled trials. Most (n = 18) of the meta-analyses were rated as critically low quality, while the remaining 1 was rated as high quality. The conclusions were discordant for all three outcome domains, even for meta-analyses reporting similar inclusion criteria. We could not perform most of the statistical tests due to the predominantly critically low quality. ConclusionThe methodological quality was so predominantly critically low that it was not possible to evaluate the association between methodological quality and reported conclusions.

The certainty of the evidence in oral health has not improved according to GRADE: a meta-epidemiological study

ObjectiveThe aim of this meta-epidemiological study was to provide an update of the certainty of the evidence in oral health by using the GRADE rating reported in oral health Cochrane systematic reviews (CSR). Study design and settingAll interventional oral health CSR published between 2003-2021 were sourced. Study characteristics were extracted at the level of the CSR and the outcome/meta-analysis. One-hundred-five CSR were eligible and analysed. ResultsAlmost a third of CSR (n=67) were excluded as a GRADE rating was not available. The most prevalent type of primary studies included in the CSR were randomized studies (93.4%) and the most used measure of effect was the risk ratio (67.3%). Overall, the certainty of the evidence according to the GRADE rating for all examined outcomes was very low/low (88%). The two most common reasons for downgrading the confidence in the evidence were study limitations (Risk of bias) and imprecision. The odds of moderate/high vs. low/very low-GRADE rating are higher for the primary compared to the secondary outcomes after adjusting for year and number of trials (OR 2.49; 95% CI: 1.09, 5.65; P=0.02). Per year (2010-2021 period) the odds of moderate/high vs. low/very low-GRADE rating decrease (OR 0.73; 95% CI: 0.60, 0.90; P=0.01), and as the number of trials per comparison increase the odds of moderate/high vs. low/very low GRADE rating increase (OR 1.13; 95% CI: 1.01, 1.25; P=0.001). ConclusionsThe certainty of the evidence in oral health when assessed with the GRADE rating remains predominantly low or very low.

Absence of small study effects in neurosurgical meta-analyses: A meta-epidemiological study

BackgroundSmall studies are prone to lower methodological quality and publication bias, and are more likely to report greater beneficial effects. A meta-epidemiological study was undertaken to investigate and quantify the impact of small study effects on meta-analyses in the neurosurgical literature. MethodsA PubMed search was used to procure meta-analyses from Journal of Neurosurgery, Neurosurgery, Spine, Acta Neurochirurgica and Journal of Neurotrauma. Outcome data were extracted from meta-analyses the effect of study size was estimated by calculating the ratio of odds ratios (RORs) between small and large studies. Results16 meta-analyses of 229 primary studies and 90,629 patients were included. All but two included pooled outcomes were significantly different from 1. On average small studies did not demonstrate greater beneficial effects, with an estimated pooled ROR of 1.32 (95% CI, 0.89 to 1.75). Stratification by meta-analysis effect size and heterogeneity yielded similar findings. ConclusionsThe absence of small study effects in meta-analyses of neurosurgical studies may reflect widespread poor quality of the neurosurgical literature affecting both large and small studies, rather than an absence of publication bias.

Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study

ObjectiveTo evaluate the agreement between diet-disease effect estimates of bodies of evidence from randomised controlled trials and those from cohort studies in nutrition research, and to investigate potential factors for...