Meta-research Study Research Articles

Authors arbitrarily used methodological approaches to analyze the quality of reporting in research reports: a meta-research study

ObjectivesMany authors used reporting checklists as an assessment tool to analyze the reporting quality of diverse types of evidence. We aimed to analyze methodological approaches used by researchers assessing reporting quality of evidence in randomized controlled trials, systematic reviews, and observational studies. Study Design and SettingWe analyzed articles reporting quality assessment of evidence with Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA), CONsolidated Standards of Reporting Trials (CONSORT), or the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) checklists published up to 18 July 2021. We analyzed methods used for assessing reporting quality. ResultsAmong 356 analyzed articles, 293 (88%) investigated a specific thematic field. The CONSORT checklist (N = 225; 67%) was most often used, in its original, modified, partial form, or its extension. Numerical scores were given for adherence to checklist items in 252 articles (75%), of which 36 articles (11%) used various reporting quality thresholds. In 158 (47%) articles, predictors of adherence to reporting checklist were analyzed. The most studied factor associated with adherence to reporting checklist was the year of article publication (N = 82; 52%). ConclusionThe methodology used for assessing reporting quality of evidence varied considerably. The research community needs a consensus on a consistent methodology for assessing the quality of reporting.

Time trends in health-related quality of life assessment and reporting within publications of oncology randomised phase III trials: a meta-research study

ObjectiveTo assess time trends in the inclusion of health-related quality of life (QoL) among study endpoints and in the reporting of QoL results in study publications, randomised phase III oncology...

Towards full clinical trial registration and results publication: longitudinal meta-research study in Northwestern and Central Switzerland

ObjectiveThe registration of clinical trials is required by law in Switzerland. We investigated (1) the proportion of registered and prospectively registered clinical trials, (2) the availability of results for ethically approved trial protocols, (3) factors associated with increased registration, and (4) reasons for non-registration.Design and settingWe included all clinical trials with mandatory prospective registration, which were approved by the ethics committee of Northwestern and Central Switzerland between January 1, 2016, and December 31, 2020.MethodsWe extracted relevant trial characteristics from the Swiss Business Administration System for Ethics Committees and systematically searched the International Clinical Trials Registry Platform and primary trial registries for corresponding registry entries. We used multivariable logistic regression to examine the association between trial characteristics and registration. We qualitatively assessed reasons for non-registration of trials through an email questionnaire for trial investigators.ResultsOf 473 included clinical trials, 432 (91%) were registered at all and 326 (69%) were prospectively registered. While the percentages of registration and prospective registration of investigator-sponsored trials increased from 85 to 93% and from 59 to 70% over 5 years, respectively, industry-sponsored trials consistently remained at a high level of prospective registration (92 to 100%). Trials with multiple centres, higher risk category, or methodological support from the local clinical trials unit were independently associated with increased registration rates. Of 103 clinical trials completed before August 2020, results were available for 70% of industry-sponsored trials and 45% of investigator-sponsored trials as peer-reviewed journal publications or in trial registries. Most common reasons for non-registration provided by investigators were lack of time or resources (53%), lack of knowledge (22%), and lack of reminders by the ethics committee (36%).ConclusionsIn Northwestern and Central Switzerland about 10% of clinical trials remained unregistered despite the obligation by law. More support for investigators and stricter enforcement by regulators are needed to improve the transparency of investigator-sponsored trials in particular.

Cannabis companies and the sponsorship of scientific research: A cross-sectional Canadian case study.

Corporations across sectors engage in the conduct, sponsorship, and dissemination of scientific research. Industry sponsorship of research, however, is associated with research agendas, outcomes, and conclusions that are favourable to the sponsor. The legalization of cannabis in Canada provides a useful case study to understand the nature and extent of the nascent cannabis industry's involvement in the production of scientific evidence as well as broader impacts on equity-oriented research agendas. We conducted a cross-sectional, descriptive, meta-research study to describe the characteristics of research that reports funding from, or author conflicts of interest with, Canadian cannabis companies. From May to August 2021, we sampled licensed, prominent Canadian cannabis companies, identified their subsidiaries, and searched each company name in the PubMed conflict of interest statement search interface. Authors of included articles disclosed research support from, or conflicts of interest with, Canadian cannabis companies. We included 156 articles: 82% included at least one author with a conflict of interest and 1/3 reported study support from a Canadian cannabis company. More than half of the sampled articles were not cannabis focused, however, a cannabis company was listed amongst other biomedical companies in the author disclosure statement. For articles with a cannabis focus, prevalent topics included cannabis as a treatment for a range of conditions (15/72, 21%), particularly chronic pain (6/72, 8%); as a tool in harm reduction related to other substance use (10/72, 14%); product safety (10/72, 14%); and preclinical animal studies (6/72, 8%). Demographics were underreported in empirical studies with human participants, but most included adults (76/84, 90%) and, where reported, predominantly white (32/39, 82%) and male (49/83, 59%) participants. The cannabis company-funded studies included people who used drugs (37%) and people prescribed medical cannabis (22%). Canadian cannabis companies may be analogous to peer industries such as pharmaceuticals, alcohol, tobacco, and food in the following three ways: sponsoring research related to product development, expanding indications of use, and supporting key opinion leaders. Given the recent legalization of cannabis in Canada, there is ample opportunity to create a policy climate that can mitigate the harms of criminalization as well as impacts of the "funding effect" on research integrity, research agendas, and the evidence base available for decision-making, while promoting high-priority and equity-oriented independent research.

The interpretation of clinical relevance in randomised clinical trials in patients with chronic low back pain: protocol for a meta-research study

Chronic low back pain (cLBP) is one of the leading worldwide causes of disability. The smallest worthwhile effect (SWE) parameter has been proposed to find a threshold of clinical relevance. Specific values of the SWE have been calculated in patients with cLBP for pain intensity, physical functioning and time to recovery for physiotherapy compared with no intervention. Our objectives are 1) To evaluate how authors have interpreted the clinical relevance of the effect of physiotherapy compared to no-intervention on pain, physical functioning and time to recovery; 2) To reinterpret the clinical relevance of these between-group differences based on the available SWE estimates; 3) To evaluate, for descriptive purposes, whether the studies are adequately powered or underpowered considering the published SWE values and a power threshold of 80%. A systematic search in Medline, PEDro, Embase and Cochrane CENTRAL will be conducted. We will search for RCT investigating the effectiveness of physiotherapy as compared to no interventions in people with cLBP. We will compare the authors' interpretation of results for clinical relevance with their results to determine if they meet their a-priori definitions. Then, we will perform a re-interpretation of the between-group differences based on SWE values published for cLBP.

'Optimal' cutoff selection in studies of depression screening tool accuracy using the PHQ-9, EPDS, or HADS-D: A meta-research study.

Optimal cutoff thresholds are selected to separate 'positive' from 'negative' screening results. We evaluated how depression screening tool studies select optimal cutoffs. We included studies from previously conducted meta-analyses of Patient Health Questionnaire-9, Edinburgh Postnatal Depression Scale, or Hospital Anxiety and Depression Scale-Depression accuracy. Outcomes included whether an optimal cutoff was selected, method used, recommendations made, and reporting guideline and protocol citation. Of 212 included studies, 172 (81%) attempted to identify an optimal cutoff, and 147 of these 172 (85%) reported one or more methods. Methods were heterogeneous with Youden's J (N=35, 23%) most common. Only 23 of 147 (16%) studies described a rationale for their method. Rationales focused on balancing sensitivity and specificity without describing why desirable. 131 of 172 studies (76%) identified an optimal cutoff other than the standard; most did not make use recommendations (N=56; 43%) or recommended using a non-standard cutoff (N=53; 40%). Only 4 studies cited a reporting guideline, and 4 described a protocol with optimal cutoff selection methods, but none used the protocol method in the published study. Research is needed to guide how selection of cutoffs for depression screening tools can be standardized and reflect clinical considerations.

Protocol for a meta-research study of protocols for diet or nutrition-related trials published in indexed journals: general aspects of study design, rationale and reporting limitations

IntroductionThe Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) reporting guideline establishes a minimum set of items to be reported in any randomised controlled trial (RCT) protocol. The Template for...

Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: cross sectional meta-research study

ObjectivesTo examine changes in completeness of reporting and frequency of sharing data, analytical code, and other review materials in systematic reviews over time; and factors associated with these changes.DesignCross sectional...

Sins of omission: A meta-research study evaluating the omission of operability in published retrospective comparisons of surgery with stereotactic body radiotherapy in patients with early-stage non-small cell lung cancer

IntroductionPatients receiving stereotactic body radiotherapy (SBRT) for early-stage non-small cell lung cancer (NSCLC) are typically inoperable, in concordance with guidelines that advocate surgical resection as preferred treatment for operable patients. This differential treatment allocation complicates retrospective comparisons of surgery with SBRT by introducing the potential for confounding by operability. MethodsPubMed was queried for manuscripts reporting primary data from retrospective comparisons of overall survival (OS) between patients undergoing surgery versus SBRT for early-stage NSCLC. Each manuscript was categorized for two outcomes: (1) whether treatment allocation was based on a determination of patient operability, and (2) whether a direct OS comparison between operable SBRT patients and surgically treated patients was included. Associations with variables of interest were measured with statistical significance prespecified at p < 0.10. ResultsFrom 3,072 manuscripts identified in our query, sixty-one analyses met screening criteria. Twenty-one (34 %) reported operability status influencing treatment allocation. These were more likely to be published in journals with a surgical focus (52 vs 20 %) and impact factor < 5 (81 vs 58 %), and to contain cohorts from institutional datasets (81 vs 55 %), and to have a radiation oncologist as first (43 vs 25 %) or senior (43 vs 28 %) author. Seven (11 %) manuscripts featured a direct OS comparison between SBRT and surgery. ConclusionNearly-two-thirds of peer-reviewed retrospective studies that have compared OS between surgery and SBRT for early-stage NSCLC lack information on patient operability status, and nearly 90% lack a direct comparison between operable SBRT patients and those receiving surgery.

Meta-research evaluating redundancy and use of systematic reviews when planning new studies in health research: a scoping review

BackgroundSeveral studies have documented the production of wasteful research, defined as research of no scientific importance and/or not meeting societal needs. We argue that this redundancy in research may to a large degree be due to the lack of a systematic evaluation of the best available evidence and/or of studies assessing societal needs.ObjectivesThe aim of this scoping review is to (A) identify meta-research studies evaluating if redundancy is present within biomedical research, and if so, assessing the prevalence of such redundancy, and (B) to identify meta-research studies evaluating if researchers had been trying to minimise or avoid redundancy.Eligibility criteriaMeta-research studies (empirical studies) were eligible if they evaluated whether redundancy was present and to what degree; whether health researchers referred to all earlier similar studies when justifying and designing a new study and/or when placing new results in the context of earlier similar trials; and whether health researchers systematically and transparently considered end users’ perspectives when justifying and designing a new study.Sources of evidenceThe initial overall search was conducted in MEDLINE, Embase via Ovid, CINAHL, Web of Science, Social Sciences Citation Index, Arts & Humanities Citation Index, and the Cochrane Methodology Register from inception to June 2015. A 2nd search included MEDLINE and Embase via Ovid and covered January 2015 to 26 May 2021. No publication date or language restrictions were applied.Charting methodsCharting methods included description of the included studies, bibliometric mapping, and presentation of possible research gaps in the identified meta-research.ResultsWe identified 69 meta-research studies. Thirty-four (49%) of these evaluated the prevalence of redundancy and 42 (61%) studies evaluated the prevalence of a systematic and transparent use of earlier similar studies when justifying and designing new studies, and/or when placing new results in context, with seven (10%) studies addressing both aspects. Only one (1%) study assessed if the perspectives of end users had been used to inform the justification and design of a new study. Among the included meta-research studies evaluating whether redundancy was present, only two of nine health domains (medical areas) and only two of 10 research topics (different methodological types) were represented. Similarly, among the included meta-research studies evaluating whether researchers had been trying to minimise or avoid redundancy, only one of nine health domains and only one of 10 research topics were represented.Conclusions that relate to the review questions and objectivesEven with 69 included meta-research studies, there was a lack of information for most health domains and research topics. However, as most included studies were evaluating across different domains, there is a clear indication of a high prevalence of redundancy and a low prevalence of trying to minimise or avoid redundancy. In addition, only one meta-research study evaluated whether the perspectives of end users were used to inform the justification and design of a new study.Systematic review registrationProtocol registered at Open Science Framework: https://osf.io/3rdua/ (15 June 2021).

Justification of research using systematic reviews continues to be inconsistent in clinical health science-A systematic review and meta-analysis of meta-research studies.

Redundancy is an unethical, unscientific, and costly challenge in clinical health research. There is a high risk of redundancy when existing evidence is not used to justify the research question when a new study is initiated. Therefore, the aim of this study was to synthesize meta-research studies evaluating if and how authors of clinical health research studies use systematic reviews when initiating a new study. Seven electronic bibliographic databases were searched (final search June 2021). Meta-research studies assessing the use of systematic reviews when justifying new clinical health studies were included. Screening and data extraction were performed by two reviewers independently. The primary outcome was defined as the percentage of original studies within the included meta-research studies using systematic reviews of previous studies to justify a new study. Results were synthesized narratively and quantitatively using a random-effects meta-analysis. The protocol has been registered in Open Science Framework (https://osf.io/nw7ch/). Twenty-one meta-research studies were included, representing 3,621 original studies or protocols. Nineteen of the 21 studies were included in the meta-analysis. The included studies represented different disciplines and exhibited wide variability both in how the use of previous systematic reviews was assessed, and in how this was reported. The use of systematic reviews to justify new studies varied from 16% to 87%. The mean percentage of original studies using systematic reviews to justify their study was 42% (95% CI: 36% to 48%). Justification of new studies in clinical health research using systematic reviews is highly variable, and fewer than half of new clinical studies in health science were justified using a systematic review. Research redundancy is a challenge for clinical health researchers, as well as for funders, ethics committees, and journals.

Checklist for Artificial Intelligence in Medical Imaging Reporting Adherence in Peer-Reviewed and Preprint Manuscripts With the Highest Altmetric Attention Scores: A Meta-Research Study.

Purpose: To establish reporting adherence to the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) in diagnostic accuracy AI studies with the highest Altmetric Attention Scores (AAS), and to compare completeness of reporting between peer-reviewed manuscripts and preprints. Methods: MEDLINE, EMBASE, arXiv, bioRxiv, and medRxiv were retrospectively searched for 100 diagnostic accuracy medical imaging AI studies in peer-reviewed journals and preprint platforms with the highest AAS since the release of CLAIM to June 24, 2021. Studies were evaluated for adherence to the 42-item CLAIM checklist with comparison between peer-reviewed manuscripts and preprints. The impact of additional factors was explored including body region, models on COVID-19 diagnosis and journal impact factor. Results: Median CLAIM adherence was 48% (20/42). The median CLAIM score of manuscripts published in peer-reviewed journals was higher than preprints, 57% (24/42) vs 40% (16/42), P < .0001. Chest radiology was the body region with the least complete reporting (P = .0352), with manuscripts on COVID-19 less complete than others (43% vs 54%, P = .0002). For studies published in peer-reviewed journals with an impact factor, the CLAIM score correlated with impact factor, rho = 0.43, P = .0040. Completeness of reporting based on CLAIM score had a positive correlation with a study's AAS, rho = 0.68, P < .0001. Conclusions: Overall reporting adherence to CLAIM is low in imaging diagnostic accuracy AI studies with the highest AAS, with preprints reporting fewer study details than peer-reviewed manuscripts. Improved CLAIM adherence could promote adoption of AI into clinical practice and facilitate investigators building upon prior works.

Establishing open science research priorities in health psychology: a research prioritisation Delphi exercise

Objective Research on Open Science practices in Health Psychology is lacking. This meta-research study aimed to identify research question priorities and obtain consensus on the Top 5 prioritised research questions for Open Science in Health Psychology. Methods and measures An international Delphi consensus study was conducted. Twenty-three experts in Open Science and Health Psychology within the European Health Psychology Society (EHPS) suggested research question priorities to create a ‘long-list’ of items (Phase 1). Forty-three EHPS members rated the importance of these items, ranked their top five and suggested their own additional items (Phase 2). Twenty-four EHPS members received feedback on Phase 2 responses and then re-rated and re-ranked their top five research questions (Phase 3). Results The top five ranked research question priorities were: 1. ‘To what extent are Open Science behaviours currently practised in Health Psychology?’, 2. ‘How can we maximise the usefulness of Open Data and Open Code resources?’, 3. ‘How can Open Data be increased within Health Psychology?’, 4. ‘What interventions are effective for increasing the adoption of Open Science in Health Psychology?’ and 5. ‘How can we increase free Open Access publishing in Health Psychology?’. Conclusion Funding and resources should prioritise the research questions identified here.

Dissemination of knowledge from Cochrane systematic reviews in public health: Cross-sectional study

Abstract Background Appropriate dissemination of public health evidence is of high importance to ensure that relevant knowledge reaches potential stakeholders and relevant population groups. A wide distrust towards science and its findings indicates that communication thereof remains below its potential. Cochrane Public Health (CPH) provides an important source of high-quality scientific evidence. This study aimed to identify (1) dissemination strategies and (2) possible stakeholders of Cochrane Public Health reviews. Methods This is a cross-sectional, meta-research study. All 68 records (reviews or protocols) listed on the CPH website https://ph.cochrane.org/cph-reviews-and-topics up to 08.03.2022 were included. Record characteristics, dissemination strategy information and potential stakeholder details were coded by one author and 10% of records were checked by another author. Data were descriptively analysed. Results 53 reviews (46 systematic reviews, 6 rapid reviews, 1 scoping review) and 15 review protocols were included. The 53 reviews were published between 2010-2022 and included 1-153 primary studies. All reviews had an open-access plain language summary (PLS) in English with translations in 3-13 other languages. Although 16 of 53 reviews and 4 of 15 protocols reported any involvement in the review process of an advisory group, only 3 of 68 records included a dissemination plan aiming to inform non-academic audiences or policy. Conclusions All identified records can be considered as relevant to a wide range of stakeholders and population groups. However, CPH reviews or protocols rarely report their dissemination strategies. It is unclear what dissemination strategies are used after CPH reviews are published. High relevance of CPH evidence for non-academic stakeholders and the general population highlights the need for adequate knowledge translation beyond academia. Key messages • Dissemination plans and implementation is rarely reported in CPH reviews. • Evidence from CPH reviews is relevant for a multitude of stakeholders.

Three out of four published systematic reviews on COVID-19 treatments were not registered and one-third of those registered were published: a meta-research study

ObjectivesThe aim of this study is to describe (1) registered and (2) published systematic reviews (SRs) on COVID-19 treatments, and to analyze (3) the proportion of publications among registered SRs and (4) the proportion of registrations among published SRs. Study Design and SettingThis meta-research study (CRD42021240423) is part of CEOsys (http://www.covid-evidenz.de/). Two reviewers identified protocols in PROSPERO (registered January 2020 to September 2020) and SRs published as preprint or peer-reviewed article in L·OVE (Living OVerview of the Evidence) COVID-19 (by May 2021). SRs of all types assessing COVID-19 treatments in humans were included. ResultsWe included 239 PROSPERO protocols and 346 SRs published in L·OVE. In both samples, the affiliation of the corresponding author with an Asian institution, standard SR as review type, and meta-analysis as synthesis method were the most frequent characteristics. Living SRs made up ≤10%. A total of 71 of 239 (29.7%) PROSPERO protocols were published as SR by February 2022, that is, after at least 17 months of follow-up (25 of 71 as preprints, 35.2%). In L·OVE, 261 of 346 (75.4%) SRs published by May 2021 were not registered in PROSPERO. ConclusionOverall, one-third PROSPERO protocols were published and three-fourth published SRs were not registered. We strongly encourage authors to register and publish their SRs promptly to reduce research waste and to allocate resources efficiently during the pandemic and beyond.

One-Third of Systematic Reviews in Rehabilitation Applied the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) System to Evaluate Certainty of Evidence: A Meta-Research Study

ObjectiveTo determine how many systematic reviews (SRs) of the literature in rehabilitation assess the certainty of evidence (CoE) and how many apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system to do this. Data SourcesFor this meta-research study, we searched PubMed and Cochrane Database of Systematic Reviews databases for SRs on rehabilitation published in 2020. Study Selection and Data ExtractionTwo reviewers independently selected the SRs and extracted the data. Reporting characteristics and appropriate use of the GRADE system were assessed. Data SynthesisThe search retrieved 827 records: 29% (239/827) SRs evaluated CoE, 68% (163/239) of which applied the GRADE system. GRADE was used by SRs of randomized controlled trials (RCTs, 88%; 144/163), non-randomized intervention studies (NRIS, 2%; 3/163), and both RCT and NRIS (10%; 16/163). In the latter case, a separate GRADE assessment according to the study design was not provided in 75% (12/16). The reasons for GRADE judgment were reported in 82% (134/163) of SRs. ConclusionsOne-third of SRs in rehabilitation assessed CoE with the GRADE system. GRADE assessment was presented transparently by most SRs. Journal editors and funders should encourage the uptake of the GRADE system when considering SRs in rehabilitation for publication. The authors should pre-define GRADE assessment in a registered and/or published protocol.

Systematic reviews are rarely used to contextualise new results—a systematic review and meta-analysis of meta-research studies

BackgroundResults of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.MethodsWe searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.ResultsWe included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.ConclusionOur systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers’ use of prior research systematically and transparently—also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.Systematic review registrationOpen Science registration number https://osf.io/8gkzu/

Lack of attention to sex and gender in periodontitis-related randomized clinical trials: A meta-research study.

This meta-research study aimed to investigate the level of compliance with the Sex and Gender Equity in Research (SAGER) Guidelines for the inclusion, analysis, and reporting of sex/gender, in periodontitis-related randomized controlled trials (RCTs). Following the inclusion of RCTs related to the treatment of periodontitis published between 2018 and 2019, we applied the SAGER checklist to assess the adherence to sex/gender reporting guidelines. We used non-parametric descriptive statistics and correlation models to test the association of the dependent outcome with other variables. One hundred and one articles were included in the analysis. The female enrolment ranged between 30% and 94%. Twenty-six studies enrolled less than 50% of female participants. The overall SAGER score (OSS) of item fulfilment ranged between 0 and 7 items with an average of 1.9 items signifying poor guideline adherence to the SAGER guidelines. These findings were not associated with the corresponding author gender (p= .623), publication year (p= .947), and funding source (p= .133). However, a significant but negative correlation with journal impact factor (r= -0.253, p= .026) was observed. Sex and gender were frequently disregarded in clinical trial reporting. This oversight might limit the understanding of sex/gender differences in periodontitis-related clinical trials.

Use of ‘Pharmaceutical services’ Medical Subject Headings (MeSH) in articles assessing pharmacists' interventions

BackgroundMedical Subject Headings (MeSH) thesaurus contribute towards efficient searching of biomedical information. However, insufficient coverage of specific fields and inaccuracies in the indexing of articles can lead to bias during literature retrieval. ObjectivesThis meta-research study aimed to assess the use of ‘Pharmaceutical Services’ MeSH terms in studies evaluating the effect of pharmacists' interventions. MethodsAn updated systematic search (Jan-2022) to gather meta-analyses comparing pharmacists' interventions vs. other forms of care was performed. All MeSH terms allocated to the MEDLINE record of each primary study included in the selected meta-analyses were systematically extracted. Terms from the ‘Pharmaceutical Services’ branch, including its descendants, as well as other 26 pharmacy-specific MeSH terms were identified. The assignment of these terms as a ‘Major MeSH’ was also evaluated. Descriptive statistics and social network analyses to evaluate the co-occurrence of the MeSH terms in the articles were conducted. Sensitivity analyses including only meta-analyses with declared objectives mentioning the words ‘pharmacist’ or ‘pharmacy’ were performed (SPSS v.24.0). ResultsOverall, 138 meta-analyses including 2012 primary articles were evaluated. A median of 15 [IQR 12–18] MeSH terms were assigned per article with a slight positive time-trend (Spearman rho = 0.193; p < 0.001). Only 36.6% (n = 736/2012) and 58.1% (n = 338/1099) of studies were indexed with one MeSH term from the ‘Pharmaceutical Services’ branch in the overall and sensitivity analyses, respectively. In <20% of cases, these terms were a ‘Major MeSH’. The pharmacy-specific term ‘Pharmacists’ was the most frequently used, yet in only 27.8% and 47.7% of articles in the original and sensitivity analyses, respectively. Social networks showed a weak association between pharmacy-specific and ‘Pharmaceutical services’ branch MeSH terms. ConclusionsThe availability of a ‘Pharmaceutical services’ branch hierarchic tree and further pharmacy-specific MeSH terms incorporated to the MeSH thesaurus in the past years is not related with accurate indexing of articles.

Code and Data Sharing Practices in the Radiology Artificial Intelligence Literature: A Meta-Research Study.

To evaluate code and data sharing practices in original artificial intelligence (AI) scientific manuscripts published in the Radiological Society of North America (RSNA) journals suite from 2017 through 2021. A retrospective meta-research study was conducted of articles published in the RSNA journals suite from January 1, 2017, through December 31, 2021. A total of 218 articles were included and evaluated for code sharing practices, reproducibility of shared code, and data sharing practices. Categorical comparisons were conducted using Fisher exact tests with respect to year and journal of publication, author affiliation(s), and type of algorithm used. Of the 218 included articles, 73 (34%) shared code, with 24 (33% of code sharing articles and 11% of all articles) sharing reproducible code. Radiology and Radiology: Artificial Intelligence published the most code sharing articles (48 [66%] and 21 [29%], respectively). Twenty-nine articles (13%) shared data, and 12 of these articles (41% of data sharing articles) shared complete experimental data by using only public domain datasets. Four of the 218 articles (2%) shared both code and complete experimental data. Code sharing rates were statistically higher in 2020 and 2021 compared with earlier years (P < .01) and were higher in Radiology and Radiology: Artificial Intelligence compared with other journals (P < .01). Original AI scientific articles in the RSNA journals suite had low rates of code and data sharing, emphasizing the need for open-source code and data to achieve transparent and reproducible science.Keywords: Meta-Analysis, AI in Education, Machine LearningSupplemental material is available for this article.© RSNA, 2022.