Meta-research Study Research Articles

Selective outcome reporting bias in randomized controlled trials on dental caries in children and adolescents: A meta-research study.

Selective Outcome Reporting (SOR) is a type of bias that potentially manifests in Randomized Controlled Trials (RCTs) and is defined as the change or omission of the primary outcome in the publication compared to the protocol. Researchers may alter outcomes to emphasize statistical significant results. We aimed to assess the prevalence of SOR in RCTs related to dental caries in Pediatric Dentistry. We conducted a search on the ClinicalTrials.gov and International Clinical Trials Registry Platform (ICTRP), using a comprehensive search strategy that included terms related to pediatric dentistry and dental caries, up to February 2023. Two independent reviewers included any trial that had two or more arms and focused on dental caries in pediatric dentistry. We excluded any registration that did not result in at least one published article. Data such as study characteristics and outcome data were extracted. The chi-square test was used with a significance level set at 5% to detect the association between SOR and the pre-specified variables. One hundred and forty-four protocols and their corresponding publications were included. SOR was observed in 58.9% (n=103), and 41.1% (n=72) having discrepancies in the primary outcome follow-up. Retrospectively registered studies accounted for 73.7%. SOR was significantly associated with discrepancies in the follow-up period (p<0.001) and with type of study design in paper (p=0.048). The high prevalence of SOR in dental caries RCTs in pediatric dentistry confirms that the issue deserves attention. Achieving transparency in the process requires the implementation of an appropriate pre-registered protocol, disclosure of deviations from it, and allowing stakeholders to compare the established protocol with the submitted documents Registration DOI: https://doi.org/10.17605/OSF.IO/5UVJ8.

COVID-19-related research data availability and quality according to the FAIR principles: A meta-research study.

According to the FAIR principles (Findable, Accessible, Interoperable, and Reusable), scientific research data should be findable, accessible, interoperable, and reusable. The COVID-19 pandemic has led to massive research activities and an unprecedented number of topical publications in a short time. However, no evaluation has assessed whether this COVID-19-related research data has complied with FAIR principles (or FAIRness). Our objective was to investigate the availability of open data in COVID-19-related research and to assess compliance with FAIRness. We conducted a comprehensive search and retrieved all open-access articles related to COVID-19 from journals indexed in PubMed, available in the Europe PubMed Central database, published from January 2020 through June 2023, using the metareadr package. Using rtransparent, a validated automated tool, we identified articles with links to their raw data hosted in a public repository. We then screened the link and included those repositories that included data specifically for their pertaining paper. Subsequently, we automatically assessed the adherence of the repositories to the FAIR principles using FAIRsFAIR Research Data Object Assessment Service (F-UJI) and rfuji package. The FAIR scores ranged from 1-22 and had four components. We reported descriptive analysis for each article type, journal category, and repository. We used linear regression models to find the most influential factors on the FAIRness of data. 5,700 URLs were included in the final analysis, sharing their data in a general-purpose repository. The mean (standard deviation, SD) level of compliance with FAIR metrics was 9.4 (4.88). The percentages of moderate or advanced compliance were as follows: Findability: 100.0%, Accessibility: 21.5%, Interoperability: 46.7%, and Reusability: 61.3%. The overall and component-wise monthly trends were consistent over the follow-up. Reviews (9.80, SD = 5.06, n = 160), articles in dental journals (13.67, SD = 3.51, n = 3) and Harvard Dataverse (15.79, SD = 3.65, n = 244) had the highest mean FAIRness scores, whereas letters (7.83, SD = 4.30, n = 55), articles in neuroscience journals (8.16, SD = 3.73, n = 63), and those deposited in GitHub (4.50, SD = 0.13, n = 2,152) showed the lowest scores. Regression models showed that the repository was the most influential factor on FAIRness scores (R2 = 0.809). This paper underscored the potential for improvement across all facets of FAIR principles, specifically emphasizing Interoperability and Reusability in the data shared within general repositories during the COVID-19 pandemic.

Differences in the reporting of conflicts of interest and sponsorships in systematic reviews with meta-analyses in dentistry: an examination of factors associated with their reporting

BackgroundReporting conflicts of interest (COI) and sources of sponsorship are of paramount importance in adequately interpreting the results of systematic reviews. Some evidence suggests that there is an influence of COI and sponsorship on the study results.The objectives of this meta-research study were twofold: (a) to assess the reporting of COI and sponsorship statements in systematic reviews published in dentistry in three sources (abstract, journal’s website and article’s full text) and (b) to assess the associations between the characteristics of the systematic reviews and reporting of COI.MethodsWe searched the PubMed database for dental systematic reviews published from database inception to June 2023. We assessed how COI and sponsorship statements were reported in the three sources. We performed a logistic regression analysis to assess the associations between the characteristics of the systematic reviews and the reporting of COI.ResultsWe assessed 924 abstracts published in PubMed and on the corresponding journals´ websites. Similarly, full texts associated with the 924 abstracts were also assessed. A total of 639 (69%) and 795 (88%) studies had no statement of COI in the abstracts on PubMed and the journal’s website, respectively. In contrast, a COI statement was reported in 801 (87%) full texts. Sponsorship statements were not reported in 911 (99%) and 847 (93%) abstracts published in PubMed and a journal´s website, respectively. Nearly two-thirds of the full-text articles (N = 607) included sponsorship statements. Journal access was significantly associated with COI statement reporting in all three sources. Open-access journals have significantly higher odds to report COI in PubMed and full-texts, while have significantly lower odds to report COI in the websites, compared with subscription or hybrid journals. type was significantly associated with COI statement reporting on the journal’s website and in the full text. Review registration based on the full text and the number of authors were significantly associated with COI statement reporting in PubMed and in the full texts. Several other variables were found to be significantly associated with COI statement reporting in one of the three sources.ConclusionsCOI and sponsorship statements seem to be underreported in the abstracts and homepage of the journals, compared to the full-texts. These results were particularly more pronounced in abstracts published in both the PubMed database and the journals’ websites. Several characteristics of systematic reviews were associated with COI statement reporting.

Resource use and costs of investigator-sponsored randomized clinical trials in Switzerland, Germany, and the United Kingdom: a metaresearch study

Background and ObjectivesConducting high-quality randomized clinical trials (RCTs) is challenging and resource intensive. Funders and academic investigators depend on limited financial resources and, therefore, need empirical data for optimal budget planning. However, current literature lacks detailed empirical data on resource use and costs of investigator-sponsored RCTs. The aim of this study is to systematically collect cost data from investigator-sponsored RCTs from Switzerland, Germany, and the United Kingdom (UK). MethodsPrincipal investigators were asked to share their RCT cost and resource use data and enter it into an online case report form. We assessed cost patterns, cost drivers, and specific cost items, examined costs by study phase (planning-, conduct-, and finalization phase), compared planned with actual RCT costs, and explored differences in cost patterns across countries, medical fields, and intervention types. ResultsWe included 93 RCTs which were initiated in Switzerland (n = 53; including eight conducted in low- and lower middle-income countries), Germany (n = 22), and the UK (n = 18). The median total trial cost in our RCT sample was $645,824 [interquartile range (IQR), $269,846–$1,577,924]. The median proportion of the total costs spent for planning phase was 27.5% [IQR, 20.6%–39.7%], for conduct phase 57.3% [IQR, 44.4%–66.3%], and for finalization phase 12.7% [IQR, 8.5%–19.3%] with little variation across countries. The items that contributed most to the total costs were protocol writing (7.2%; IQR 3.8%–10.6%), data management (5.0%; IQR 2.2%–8.1%) and follow-up (4.5%; IQR 2.3%–8.4%). Of the 66 RCTs with an available original budget, 46 (69.7%) exceeded the budget by over 50%. Use of routinely collected data to assess primary outcomes was independently associated with lower per patient- and lower total trial costs. ConclusionOver a quarter of total trial costs were incurred in the planning phase, which is typically not fully funded. Two-thirds of RCTs exceeded their budget by more than 50%. Investigators and funders should consider empirical cost data to improve budgeting and funding practices.

Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a Qualitative Study.

Quality assessment of diagnostic accuracy studies (QUADAS), and more recently QUADAS-2, were developed to aid the evaluation of methodological quality within primary diagnostic accuracy studies. However, its current form, QUADAS-2 does not address the unique considerations raised by artificial intelligence (AI)-centered diagnostic systems. The rapid progression of the AI diagnostics field mandates suitable quality assessment tools to determine the risk of bias and applicability, and subsequently evaluate translational potential for clinical practice. We aim to develop an AI-specific QUADAS (QUADAS-AI) tool that addresses the specific challenges associated with the appraisal of AI diagnostic accuracy studies. This paper describes the processes and methods that will be used to develop QUADAS-AI. The development of QUADAS-AI can be distilled into 3 broad stages. Stage 1-a project organization phase had been undertaken, during which a project team and a steering committee were established. The steering committee consists of a panel of international experts representing diverse stakeholder groups. Following this, the scope of the project was finalized. Stage 2-an item generation process will be completed following (1) a mapping review, (2) a meta-research study, (3) a scoping survey of international experts, and (4) a patient and public involvement and engagement exercise. Candidate items will then be put forward to the international Delphi panel to achieve consensus for inclusion in the revised tool. A modified Delphi consensus methodology involving multiple online rounds and a final consensus meeting will be carried out to refine the tool, following which the initial QUADAS-AI tool will be drafted. A piloting phase will be carried out to identify components that are considered to be either ambiguous or missing. Stage 3-once the steering committee has finalized the QUADAS-AI tool, specific dissemination strategies will be aimed toward academic, policy, regulatory, industry, and public stakeholders, respectively. As of July 2024, the project organization phase, as well as the mapping review and meta-research study, have been completed. We aim to complete the item generation, including the Delphi consensus, and finalize the tool by the end of 2024. Therefore, QUADAS-AI will be able to provide a consensus-derived platform upon which stakeholders may systematically appraise the methodological quality associated with AI diagnostic accuracy studies by the beginning of 2025. AI-driven systems comprise an increasingly significant proportion of research in clinical diagnostics. Through this process, QUADAS-AI will aid the evaluation of studies in this domain in order to identify bias and applicability concerns. As such, QUADAS-AI may form a key part of clinical, governmental, and regulatory evaluation frameworks for AI diagnostic systems globally. DERR1-10.2196/58202.

Data sharing policies across health research globally: Cross-sectional meta-research study.

Data sharing improves the value, synthesis, and integrity of research, but rates are low. Data sharing might be improved if data sharing policies were prominent and actionable at every stage of research. We aimed to systematically describe the epidemiology of data sharing policies across the health research lifecycle. This was a cross-sectional analysis of the data sharing policies of the largest health research funders, all national ethics committees, all clinical trial registries, the highest-impact medical journals, and all medical research data repositories. Stakeholders' official websites, online reports, and other records were reviewed up to May 2022. The strength and characteristics of their data sharing policies were assessed, including their policies on data sharing intention statements (a.k.a. data accessibility statements) and on data sharing specifically for coronavirus disease studies. Data were manually extracted in duplicate, and policies were descriptively analysed by their stakeholder and characteristics. Nine hundred and thirty-five eligible stakeholders were identified: 110 funders, 124 ethics committees, 18 trial registries, 273 journals, and 410 data repositories. Data sharing was required by 41% (45/110) of funders, no ethics committees or trial registries, 19% (52/273) of journals and 6% (24/410) of data repositories. Among funder types, a higher proportion of private (63%, 35/55) and philanthropic (67%, 4/6) funders required data sharing than public funders (12%, 6/49). Data sharing requirements, and even recommendations, were insufficient across health research. Where data sharing was required or recommended, there was limited guidance on implementation. We describe multiple pathways to improve the implementation of data sharing. Public funders and ethics committees are two stakeholders with particularly important untapped opportunities.

Reporting quality of scoping reviews in endodontics: A meta-research study.

To evaluate the reporting quality of Scoping Reviews (ScRs) in endodontics according to the PRISMA Extension Checklist for Scoping Reviews (PRISMA-ScR) and to analyse their association with a range of publication and methodological/reporting characteristics. Pubmed, Scopus, and Web of Science databases were searched up to 31 January 2024 to identify scoping reviews in the field of endodontics. An additional search was performed in three leading endodontic journals. Study selection and appraising the quality of the studies was carried out independently by two reviewers. Each of the 20 PRISMA-ScR items were allocated a score of either 0, 0.5 or 1 to reflect the completeness of the reporting. An item-specific and overall percentage reporting quality score was calculated and reported through descriptive statistics across a range of publication, as well as methodological/reporting characteristics. A univariable and multivariable quantile regression was performed to identify the effect of publication and methodological/reporting characteristics (year of publication, journal, inclusion of an appropriate reporting guideline, and study registration) on the overall percentage reporting quality score. Association of reporting quality score with publication characteristics was then investigated. A total of 40 ScRs were identified and included for appraisal. Most of the studies were published from 2021 onwards. The overall median reporting quality score was 86%. The most frequent items not included in the studies were: a priori protocol registration (22/40 compliant; 55%), and reporting of funding (16/40 compliant; 40%). Other key elements that were inadequately reported were the abstract (7/40 compliant; 18%), the rationale and justification of the ScR (21/40 compliant; 52%) and the objectives of the study (18/40 compliant; 45%). Studies that adhered to appropriate reporting guidelines were associated with greater reporting quality scores (β-coefficient: 10; 95%CI: 1.1, 18.9; p = .03). ScRs with protocols registered a priori had significantly greater reporting quality scores (β-coefficient: 12.5; 95%CI: 6.1, 18.9; p < .001), compared with non-registered reviews. The reporting quality of the ScRs in endodontics varied and was greater when the ScR protocols were registered a priori and when the authors adhered to reporting guidelines.

Most physical interventions for musculoskeletal pain in children and adolescents cannot be reproduced in clinical practice: a meta-research study of randomized clinical trials

IntroductionConservative treatments such as physical therapies are usually the most indicated for the management of musculoskeletal pain; therefore, a detailed description of interventions enables the reproducibility of interventions in clinical practice and future research. The objective of this study is to evaluate the description of physical interventions for musculoskeletal pain in children and adolescents.MethodsWe considered randomized controlled trials that included children and adolescents between 4 and 19 years old with acute or chronic/persistent musculoskeletal pain. We included physical therapies related to all types of physical modalities aimed at reducing the intensity of pain or disability in children and adolescents with musculoskeletal pain. The description of interventions was assessed using the Template for Intervention Description and Replication (TIDieR) checklist. We performed electronic searches in the following databases: CENTRAL, MEDLINE, EMBASE, CINAHL, PsyINFO and PEDro up to April 2024. The description of physical interventions was presented using frequencies, percentages and 95% confidence intervals (CIs) of the TIDieR checklist items described in each study. We also calculated the total TIDieR score for each study and presented these data as mean and standard deviation.ResultsWe included 17 randomized controlled trials. The description measured through the TIDieR checklist scored an average of 11 (5.2) points out of 24. The item of the TIDieR that was most described was item 1 (brief name) and most absent was item 10 (modifications).ConclusionThe descriptions of physical interventions for the treatment of musculoskeletal pain in children and adolescents are partially described, indicating the need for strategies to improve the quality of description to enable true clinical reproducibility.

A Critical Appraisal of Reporting in Randomized Controlled Trials Investigating Osteopathic Manipulative Treatment: A Meta-Research Study.

Background/Objectives: In osteopathy, it becomes necessary to produce high-quality evidence to demonstrate its effectiveness. The aim of this meta-research study is to assess the reporting quality of RCTs published in the osteopathic field. Methods: The protocol was preliminarily registered on the "Open Science Framework (OSF)" website. For reporting, we considered the PRISMA 2020 checklist. We included all the RCTs, published between 2011 and 2023, investigating the effectiveness of Osteopathic Manipulative Treatment (OMT) in any possible condition. The search process was conducted on four major biomedical databases including PubMed, Central, Scopus and Embase. A data extraction form was implemented to collect all relevant information. The completeness of reporting was calculated as the percentage of adherence to the CONSORT checklist; the Cochrane ROB 2 tool was considered to assess the risk of bias (RoB) in the following five major domains: randomization (D1), interventions (D2), missing data (D3), outcome measurement (D4), selective reporting (D5). Results: A total of 131 studies were included and the overall adherence was 57%, with the worst section being "other information" (42%). Studies with a lower RoB showed higher adherence to the CONSORT. The "results" section presented the highest differences as follows: D1 (-36.7%), D2 (-27.2%), D3 (-21.5%) and D5 (-25.5%). Significant correlations were also found between the preliminary protocol registration, higher journal quartile, publication in hybrid journals and the completeness of reporting (β: 19.22, CI: 14.45-24.00, p < 0.001; β: 5.41; CI: 2.80-8.02, p ≤ 0.001; β: 5.64, CI: 1.06-10.23, p = 0.016, respectively). Conclusions: The adherence to the CONSORT checklist in osteopathic RCTs is lacking. An association was found between a lower completeness of reporting and a higher RoB, a good journal ranking, publication in hybrid journals and a prospective protocol registration. Journals and authors should adopt all the strategies to adhere to reporting guidelines to guarantee generalization of the results arising from RCTs.

Assessing transparency practices in dental randomized controlled trials

BackgroundTo evaluate transparency practices in randomized controlled trials (RCTs) in dentistry.MethodsThis meta-research study included RCTs in dentistry regardless of topic, methods, or level of detail reported. Only studies in English were considered. We searched PubMed for RCTs in dentistry published in English from December 31, 2016, to December 31, 2021. The screening was performed in duplicate, and data extracted included journal and author details, dental specialty, protocol registration, data and code sharing, conflict of interest declaration, and funding information. A descriptive analysis of the data was performed. We generated maps illustrating the reporting of transparency items by country of the corresponding author and a heat table reflecting reporting levels by dental specialty.ResultsA total of 844 RCTs were included. Only 12.86% of studies reported any information about data and code sharing. Protocol registration was reported for 50.36% of RCTs. Conflict of interest (83.41%) and funding (71.68%) declarations were present in most studies. Conflicts of interest and funding were consistently reported regardless of country or specialty, while data and code sharing had a low level of reporting across specialties, as well as low dissemination across the world. Protocol registration exhibited considerable variability.ConclusionsConsidering the importance of RCTs for evidence-based dentistry, it is crucial that everyone who participates in the scientific production and dissemination process actively and consistently promotes adherence to transparent scientific standards, particularly registration of protocols, and sharing of data and code.

The policies on the use of large language models in radiological journals are lacking: a meta-research study

ObjectiveTo evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence.MethodsIn this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated.ResultsThe LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001).ConclusionThe presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency.Critical relevance statementIt may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals.Key PointsThe policies on LLM use in radiological journals are unexplored.Some of the radiological journals presented policies on LLM use.A shared complete reporting guideline for LLM use is desired.Graphical

Characteristics and completeness of reporting of systematic reviews of prevalence studies in adult populations: a metaresearch study

ObjectivesThe Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, first published in 2009, has been widely endorsed and compliance is high in systematic reviews (SRs) of intervention studies. SRs of prevalence studies are increasing in frequency, but their characteristics and reporting quality have not been examined in large studies. Our objectives were to describe the characteristics of SRs of prevalence studies in adults, evaluate the completeness of reporting, and explore study-level characteristics associated with the completeness of reporting. Study Design and SettingWe did a metaresearch study. We searched 5 databases from January 2010 to December 2020 to identify SRs of prevalence studies in adult populations. We used the PRISMA 2009 checklist to assess completeness of reporting and recorded additional characteristics. We conducted a descriptive analysis of review characteristics and linear regression to assess the relationship between compliance with PRISMA and publication characteristics. ResultsWe included 1172 SRs of prevalence studies. The number of reviews increased from 25 in 2010 to 273 in 2020. The median PRISMA score for SRs without meta-analysis was 17.5 of a maximum of 23, and for SRs with meta-analysis, 22 of a maximum of 25. Completeness of reporting, particularly for key items in the methods section, was suboptimal. SRs that included a meta-analysis or reported using a reporting or conduct guideline were the factors most strongly associated with increased compliance with PRISMA 2009. ConclusionReporting of SRs of prevalence was adequate for many PRISMA items. Nonetheless, this study highlights aspects for which special attention is needed. Development of a specific tool to assess the risk of bias in prevalence studies and an extension to the PRISMA statement could improve the conduct and reporting of SRs of prevalence studies.

Substantial discrepancies exist between registered protocol and published manuscript in trials on exercise interventions for chronic low back pain: a metaresearch study

ObjectivesReporting bias, prevalent in biomedical fields, can undermine evidence credibility. Our objective was to evaluate the proportion of discrepancies between registered protocols and published manuscripts in randomized controlled trials (RCTs) on exercise interventions for patients with chronic low back pain (CLBP). Study Design and SettingWe conducted a cross-sectional meta-research study, starting from the 2021 “Exercise therapy for CLBP” Cochrane Review. We selected all RCTs reporting a protocol registration on a primary register of the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) or in ClinicalTrials.gov. We extracted data from both registered protocol and published manuscript of RCTs, collecting recruitment and administrative information (eg, record dates) and details of trial characteristics (eg, outcomes, arms, statistical analysis plan details [SAPs]). Independent pairs of reviewers assessed discrepancies between registered protocol and published manuscript for the reporting of primary and secondary outcomes domains, measurement instruments, time-points, number of arms and SAPs(if attached). Outcome discrepancies were characterized as addition, omission, upgrade or downgrade. ResultsWe included 116 RCTs reporting an available protocol registration. Overall, 100 RCTs (86.2%) distinguished between primary and secondary outcomes. Of these, 39 RCTs (39.0%) reported one or more discrepancies in primary outcomes, and 78 RCTs (78.0%) reported one or more discrepancies in secondary outcomes. Focusing on discrepancies for the primary outcome, 64.5% of added, upgraded or downgraded outcomes favored statistically significant effects. Few RCTs (n = 6) reported discrepancies in the number of arms. SAPs were poorly reported in the registered protocols (n = 3) for being compared to the publications. ConclusionWe found substantial outcome discrepancies comparing registered protocols and published manuscripts in RCTs assessing exercise interventions for patients with CLBP, with some impacting the statistical significance of the effects. Readers are encouraged to approach RCTs results in this field with caution.

Primary outcomes and characteristics of clinical trial registries (up to October 2021) on COVID-19 vaccines.

To analyse the general and primary outcome-related characteristics of clinical trials protocols on COVID-19 vaccines. A meta-research study. A search for clinical trial protocols on COVID-19 vaccines was conducted on the ClinicalTrials.gov platform. We considered all protocols of comparative trials registered up to October 26, 2021. Two hundred and eighty-two trials were analysed. The median expected trial duration was 445 days (interquartile range [IQR] = 225), and the median target sample size was 420 participants (IQR = 1638). A retrospective registry (after the start date) was observed for 42.55% of the trials. Randomization procedures were planned by 84.75% and full-blinding procedures by 34.75% of the 282 trials. Most trials were labelled as active or still recruiting, and 14 trials (5%) were completed. None of the 14 trials labelled as completed on our search date had results available. Industry funding was reported by 198 trials (70.2%). Most studies declared more than one primary outcome, usually a safety or immunogenicity outcome, and 59 studies (20.9%) had at least one primary efficacy outcome. The description of the primary efficacy outcomes was limited in most cases, referred to as a non-specified 'efficacy' outcome (18.6%) or described as 'COVID-19 cases' (32.2%). the primary outcomes of clinical trials on COVID-19 vaccines are poorly described, and the registers provide insufficient information about them. The registry was retrospectively fulfilled for many trials, which may lead to bias and research waste. Outcomes were generically described and did not provide transparent information for replication in practice, further trials or meta-analyses.

Identification success rates in the post-Spanish Civil War mass graves located in the cemetery of Paterna, Spain: Meta-research on 15 mass graves with 933 subjects

Spain suffered a Civil War between 1936 and 1939 that ended with the victory of the National Forces led by General Franco. Once the Spanish Civil War ended, 2238 subjects were executed and buried in several mass graves in the Cemetery of Paterna, one of Spain's largest mass grave sites. Efforts to locate and identify all the victims of the mass graves of the Paterna cemetery are ongoing, but the actual data of the percentage of DNA identifications remains uncertain. Following this, we conducted a meta-research study including 15 mass graves and 933 subjects to determine the DNA identification rates in the mass graves of the Paterna cemetery. We found that the total proportion of identified subjects in the mass graves was 15.9 % (95 % CI: 10.0–22.9). Moreover, we found that the model between the identification success rate (ISR) and the number of relatives that donated DNA (NRTDD) in the mass graves of the cemetery of Paterna was ISR = NRTDD−0.424. Results obtained about the proportion of identified subjects and the model between the ISR and the NRTDD imply the need for a scientific reflection between all the research groups involved in the identification tasks to modify deficiencies and update identification protocols to obtain better future results.

Health-related quality of life (QoL) in randomized phase III trials in oncology: Association between results of QoL, results of primary endpoint and drug approval.

11109 Background: scientific community and regulatory agencies have shown a growing interest on QoL, although the inclusion and reporting of QoL analysis in clinical trials is still suboptimal. Methods: we previously published a meta-research study of phase III randomized clinical trial (RCT) in patients with solid neoplasms treated with systemic therapy, published from 2012 to 2021 (BMJ Oncology 2023;2:e000021). For the present analysis, we selected the RCT conducted in the advanced setting, integrating the database with results of QoL and of primary endpoints, and with info about regulatory approval. The main outcome was the analysis of correlation of QoL results with study primary endpoint (EP1), namely overall survival (OS) or progression free survival (PFS). Among secondary outcomes, the availability of QoL results was reported for treatments approved by EMA/FDA, with description of time-trends. Results: 592phase III RCTpublished from 2012 to 2021 were included: 322 (54.4%) published in 2012-2016 and 270 (45.6%) in 2017-2021. 151 RCT (25.5%) were conducted in gastro-intestinal cancers, 138 (23.3%) in thoracic cancers, 71 (12%) in breast cancer, 79 (13.3%) in genito-urinary cancers. Experimental treatment was chemotherapy in 322 studies (54.4%), targeted therapies in 331 (55.9%), immunotherapy in 94 (15.9%) and hormone therapy in 52 (8.8%). OS was the EP1 in 298 clinical trials (50.3%) and PFS in 304 clinical trials (51.4%), with an overlap for 79 studies (13.3%) with multiple primary endpoints. 124 RCT (41.6%) with EP1 OS reported a positive result in EP1. Among these, QoL analysis was positive for experimental treatment in 62 studies (50%), without statistically significant difference or unfavourable in 30 (24.2%) and not available in 32 (25.8%). In the 182 studies (59.5%) with EP1 PFS and a positive result in EP1, QoL analysis was positive for experimental arm in 77 studies (42.3%), without statistically significant difference or unfavourable in 49 (26.9%) and not available in 56 (30.8%). FDA drug approvals were reported for 143 studies (24.2%). Among them, QoL results were positive for experimental arm in 101 studies (70.6%), negative in 19 (13.3%), absent in 23 (16.1%). Similarly, 142 studies (24%) were associated to EMA approval: positive QoL in 101 studies (71.1%), negative in 21 (14.8%) and absent in 20 (14.1%). The percentage of FDA and EMA approvals associated with the availability of positive QoL data increased from 2012-2016 to 2017-2021. Namely, the proportion of approvals with available QoL positive results increased from 56.5% to 81.5% (p&lt;0.001) among FDA approvals, and from 55.4% to 84.4% (p&lt;0.001) among EMA approvals. Conclusions: in many cases, a positivity in OS or PFS is not accompanied by the demonstration of QoL benefit. The temporal trend of positive QoL results among treatments approved by regulatory agencies is encouraging.

Workflow for detecting biomedical articles with underlying open and restricted-access datasets.

To monitor the sharing of research data through repositories is increasingly of interest to institutions and funders, as well as from a meta-research perspective. Automated screening tools exist, but they are based on either narrow or vague definitions of open data. Where manual validation has been performed, it was based on a small article sample. At our biomedical research institution, we developed detailed criteria for such a screening, as well as a workflow which combines an automated and a manual step, and considers both fully open and restricted-access data. We use the results for an internal incentivization scheme, as well as for a monitoring in a dashboard. Here, we describe in detail our screening procedure and its validation, based on automated screening of 11035 biomedical research articles, of which 1381 articles with potential data sharing were subsequently screened manually. The screening results were highly reliable, as witnessed by inter-rater reliability values of ≥0.8 (Krippendorff's alpha) in two different validation samples. We also report the results of the screening, both for our institution and an independent sample from a meta-research study. In the largest of the three samples, the 2021 institutional sample, underlying data had been openly shared for 7.8% of research articles. For an additional 1.0% of articles, restricted-access data had been shared, resulting in 8.3% of articles overall having open and/or restricted-access data. The extraction workflow is then discussed with regard to its applicability in different contexts, limitations, possible variations, and future developments. In summary, we present a comprehensive, validated, semi-automated workflow for the detection of shared research data underlying biomedical article publications.

Reporting completeness of nutrition and diet-related randomised controlled trials protocols

Background and AimsThere is a need to consolidate reporting guidance for nutrition randomised controlled trial (RCT) protocols. The reporting completeness in nutrition RCT protocols and study characteristics associated with adherence to SPIRIT and TIDieR reporting guidelines are unknown. We, therefore, assessed reporting completeness and its potential predictors in a random sample of published diet and nutrition-related RCT protocols. MethodsWe conducted a meta-research study of 200 nutrition and diet-related RCT protocols published in 2019 and 2021 (aiming to consider periods before and after the start of the COVID pandemic). Data extraction included bibliometric information, general study characteristics, compliance with 122 questions corresponding to items and subitems in the SPIRIT and TIDieR checklists combined, and mention to these reporting guidelines in the publications. We calculated the proportion of protocols reporting each item and the frequency of items reported for each protocol. We investigated associations between selected publication aspects and reporting completeness using linear regression analysis. ResultsThe majority of protocols included adults and elderly as their study population (n=73; 36.5%), supplementation as intervention (n=96; 48.0%), placebo as comparator (n=89; 44.5%), and evaluated clinical status as the outcome (n=80; 40.0%). Most protocols described a parallel RCT (n=188; 94%) with a superiority framework (n=141; 70.5%). Overall reporting completeness was 52.0% (SD=10.8%). Adherence to SPIRIT items ranged from 0% (n=0) (data collection methods) to 98.5% (n=197) (eligibility criteria). Adherence to TIDieR items ranged from 5.5% (n=11) (materials used in the intervention) to 98.5% (n=197) (description of the intervention). The multivariable regression analysis suggests that a higher number of authors [β=0.53 (95%CI: 0.28-0.78)], most recent published protocols [β=3.19 (95%CI: 0.24-6.14)], request of reporting guideline checklist during the submission process by the journal [β=6.50 (95%CI: 2.56-10.43)] and mention of SPIRIT by the authors [β = 5.15 (95%CI: 2.44-7.86)] are related to higher reporting completeness scores. ConclusionsReporting completeness in a random sample of 200 diet or nutrition-related RCT protocols was low. Number of authors, year of publication, self-reported adherence to SPIRIT, and journals’ endorsement of reporting guidelines seem to be positively associated with reporting completeness in nutrition and diet-related RCT protocols.

Only half of the authors of overviews of exercise-related interventions use some strategy to manage overlapping primary studies—a metaresearch study

ObjectivesThe conduct of systematic reviews (SRs) and overviews share several similarities. However, because the unit of analysis for overviews is the SRs, there are some unique challenges. One of the most critical issues to manage when conducting an overview is the overlap of data across the primary studies included in the SRs. This metaresearch study aimed to describe the frequency of strategies to manage the overlap in overviews of exercise-related interventions. Study Design and SettingA systematic search in MEDLINE (Ovid), Embase (Ovid), Cochrane Library, Epistemonikos, and other sources was conducted from inception to June 2022. We included overviews of SRs that considered primary studies and evaluated the effectiveness of exercise-related interventions for any health condition. The overviews were screened by two authors independently, and the extraction was performed by one author and checked by a second. We found 353 overviews published between 2005 and 2022 that met the inclusion criteria. ResultsOne hundred and sixty-four overviews (46%) used at least one strategy to visualize, quantify, or resolve overlap, with a matrix (32/164; 20%), absolute frequency (34/164; 21%), and authors' algorithms (24/164; 15%) being the most used methods, respectively. From 2016 onwards, there has been a trend toward increasing the use of some strategies to manage overlap. Of the 108 overviews that used some strategy to resolve the overlap, ie, avoiding double or multiple counting of primary study data, 79 (73%) succeeded. In overviews where no strategies to manage overlap were reported (n = 189/353; 54%), 16 overview authors (8%) recognized this as a study limitation. ConclusionAlthough there is a trend toward increasing its use, only half of the authors of the overviews of exercise-related interventions used a strategy to visualize, quantify, or resolve overlap in the primary studies’ data. In the future, authors should report such strategies to communicate more valid results.

Catalysing global surgery: a meta-research study on factors affecting surgical research collaborations with Africa.

In December 2019, the COVID-19 pandemic highlighted the urgent need for rapid collaboration, research, and interventions. International research collaborations foster more significant responses to rapid global changes by enabling international, multicentre research, decreasing biases, and increasing study validity while reducing overall research time and costs. However, there has been low uptake of collaborative research by African institutions and individuals. To systematically review facilitating factors and challenges to collaborative surgical research studies conducted in Africa. A meta-research review using PubMed®/MEDLINE and Embase on surgical collaboration in Africa from 1st of January 2011 to 31st of September 2021 in accordance to PRISMA guidelines. Surgical studies by collaborative groups involving African authors and sites were included (55 papers). Data on the study period, geographical regions, and research scope, facilitating factors, and challenges were extracted from the studies retrieved from the search. Most of the collaborations in Africa occurred with European institutions (76%). Of the 54 African countries, 63% (34/54) participated in surgical collaborations. The highest collaboration frequency occurred in South Africa (11%) and Nigeria (8%). However, most publications originated from Eastern Africa (43%). Leveraging synergies between high- and low- to middle-income countries (LMICs), well-defined structures, and secure data platforms facilitated collaboration. However, the underrepresentation of collaborators from LMICs was a significant challenge. Available literature provides critical insights into the facilitating factors and challenges of research collaboration with Africa. However, there is a need for a detailed prospective study to explore the themes highlighted further. PROSPERO 2022 CRD42022352115 .