Can ChatGPT Recognize Its Own Writing in Scientific Abstracts?
Background: With the growing use of generative AI in scientific writing, distinguishing between AI-generated and human-authored content has become a pressing challenge. It remains unclear whether ChatGPT (OpenAI, San Francisco, CA) can accurately and consistently recognize its own output.Methods: We randomly selected 100 research articles published in 2000, before the advent of generative AI, from 10 high-impact internal medicine journals. For each article, a structured abstract was generated using ChatGPT-4.0 based on the full PDF. The original and AI-generated abstracts (n = 200) were then evaluated twice by ChatGPT-4.0, which was asked to rate the likelihood of authorship on a 0-10 scale (0 = definitely human, 10 = definitely ChatGPT, 5 = undetermined). Classifications of 0-4 were considered human, and 6-10 were considered AI generated.Results: Misclassification rates were high in both rounds (49% and 47.5%). No abstract received a score of 5. Score distributions overlapped substantially between groups, with no statistically significant difference (Wilcoxon p-value = 0.93 and 0.21). Cohen’s kappa for binary classification was 0.33 (95% CI: 0.19-0.46) and weighted kappa on the 0-10 scale was 0.24 (95% CI: 0.15-0.34), both reflecting poor agreement.Conclusion: ChatGPT-4.0 cannot reliably identify whether a scientific abstract was written by itself or by humans. More robust external tools are needed to ensure transparency in academic authorship.
- Research Article
58
- 10.1136/bmj.k3654
- Oct 3, 2018
- BMJ
ObjectivesTo determine the role of academic authors, funders, and contract research organisations in industry funded trials of vaccines, drugs, and devices and to determine lead academic authors’ experiences with industry...
- Research Article
- 10.4081/gc.2020.9227
- Oct 1, 2020
- Geriatric Care
To protect appropriate authorship, the International Committee of Medical Journal Editors (ICMJE) formulated a guideline on authorship. Researchers not fulfilling these criteria and still enlisted as author are seen as honorary authors (HA). The objective of this study is to assess authorship decision making and the proportion of HA in journals in the field of geriatrics and gerontology. Corresponding authors of six highimpact journals in geriatrics and gerontology were sent a survey. The survey consisted of three parts: i) demographics of the respondent; ii) awareness of authorship guidelines; and iii) authorship decisions made for the paper they are authors of. Respondents were also asked if one of their coauthors does not deserve authorship. This is defined as self-perceived HA. Furthermore, respondents were asked if any of their co-authors only performed tasks from a list of non-authorship tasks. This is defined as ICMJE-defined HA. Of the 1592 contacted authors, 528 filled in a survey (response rate 33.2%). 84.4% was aware of the ICMJE-guidelines, but 44.2% was unaware of the issue of HA. The proportion of self-perceived HA was 12.7%. Independent factors associated with more self-perceived HA were having a senior member automatically enlisted as coauthor [odds ratio (OR) 3.4, 95%confidence interval (CI) 1.8 to 6.4] and have gotten the suggestion to include an HA (OR 11.1, 95% CI 4.4 to 27.9). The proportion of ICMJE defined HA was 39.3%. The journal surveyed (OR 1.2, 1.0 to 1.3) was associated with more, and awareness of the ICMJE-guidelines (OR 0.5, 95% CI 0.3 to 0.9) was associated with less ICMJE-defined HA respectively. Having a senior member automatically enlisted as co-author (OR 2.1 95% CI 1.3 to 3.4) and having anyone suggest to include an HA (OR 4.8 95% CI 1.8 to 12.8) were also associated with more ICMJE-defined HA. More than one out of ten of the corresponding authors thinks that based on the ICMJE-guidelines, one or more of their coauthors did not deserve authorship. A stricter journal policy and more awareness of the ICMJE-guidelines could help reduce the proportion of HA.
- Research Article
21
- 10.1192/bjo.2017.8
- Feb 22, 2018
- BJPsych open
There is a clear gender gap in scientific authorship. Although the proportions of female authors in medicine and psychiatry have increased over the past decades, women are still underrepresented. Aims To analyse authorship gender trends in eating disorder research. First and last author gender in research articles on eating disorders during the period 1997-2016 were assessed in eating disorder specialty journals, high-impact psychiatry journals and high-impact clinical psychology journals. The total number of papers on eating disorders increased substantially over the observation period, although a decrease was observed in high-impact psychiatry journals. Female authorship increased in both specialty journals and high-impact psychiatry journals. Authors were significantly less likely to be female in high-impact psychiatry and clinical psychology journals than in speciality journals. Eating disorder research has been increasingly allocated to specialty journals over the past 20 years. A consistent gender gap between specialty and high-impact journals exists. Declaration of interest C.M.B is a grant recipient from Shire Pharmaceuticals, Inc. and has participated as a member of their scientific advisory board. These positions are unrelated to the content of this article.
- Research Article
48
- 10.1111/jnu.12093
- Jun 13, 2014
- Journal of Nursing Scholarship
The purposes of this study were to (a) assess the prevalence of articles with honorary authors and ghost authors in 10 leading peer-reviewed nursing journals between 2010 to 2012; (b) compare the results to prevalence reported by authors of articles published in high-impact medical journals; and (c) assess the experiences of editors in the International Academy of Nursing Editors with honorary and guest authorship. Corresponding authors of articles published in 10 nursing journals between 2010 and 2012 were invited to complete an online survey about the contributions of coauthors to see if the International Committee of Medical Journal Editors () criteria for authorship were met. Additionally, members of the International Academy of Nursing Editors were invited to complete an online survey about their experiences in identifying honorary or ghost authors in articles submitted for publication. The prevalence of articles published in 10 nursing journals with honorary authors was 42%, and the prevalence of ghost authorship was 27.6%. This is a greater prevalence than what has been reported among medical journals. Qualitative data yielded five themes: lack of awareness around the rules for authorship; acknowledged need for debate, discussion, and promotion of ethical practice; knowingly tolerating, and sometimes deliberately promoting, transgressions in practice; power relations and expectations; and avoiding scrutiny. Among the 60 respondents to the editor survey, 22 (36.7%) reported identifying honorary authors and 13 (21.7%) reported ghost authors among papers submitted to their publications. Inappropriate authorship is a significant problem among scholarly nursing publications. If nursing scholarship is to maintain integrity and be considered trustworthy, and if publications are to be a factor in professional advancement, editors, nursing leaders, and faculty need to disseminate and adhere to ethical authorship practices.
- Research Article
2
- 10.17294/2330-0698.1735
- Jul 27, 2020
- Journal of Patient-Centered Research and Reviews
Because financial conflicts of interest (FCOIs) may potentially influence patient care, hospital drug formularies, and treatment guidelines, it is important that these are disclosed. The purpose of this observational study was to quantify the changes in FCOI among U.S.-based academic authors in industry-sponsored oncology trials after a high-impact publication. A list of all U.S.-based academic authors (authors) of industry-sponsored solid tumor clinical trials published between August 1, 2014, and December 31, 2015, in 6 high-impact journals (New England Journal of Medicine, Nature, Science, Lancet Oncology, Journal of Clinical Oncology, and Cancer Discovery) was assembled. Studies were limited to solid tumor oncology trials. After all authors were identified, direct and research funding was tabulated from CMS Open Payments for the year prior (Ypre) and the first 3 years following publication (Y1, Y2, Y3) in the high-impact journal. Summary statistics were tabulated and repeated-measures linear mixed-effects regression models were fit to examine changes after publication. A total of 102 publications with a total of 620 authors were identified. No FCOI was declared by 11, 12, 21, and 24 authors in Ypre, Y1, Y2, and Y3, respectively. In Ypre, Y1, Y2, and Y3: median FCOI for direct payments was $16,702 (range: $0-$3,180,356), $20,830 (range: $0-$3,180,356), $22,031 (range: $0-$920,746), and $21,356 (range: $0-$920,707), respectively; while median research funding was $559,202 (range: $0-$19,973,818), $505,031 (range $0-$19,920,452), $502,726 (range: $0-$15,729,776), and $497,342 (range: $0-$43,036,716), respectively. There were nonsignificant increases in total direct payments and total direct payments received from the sponsor (P>0.0125 for both) and statistically significant decreases in total associated research funding and total research funding from the research sponsor in Y1, Y2, and Y3 as compared to Ypre (P<0.0001 for both). After publication of an industry-sponsored solid tumor clinical trial in a high-impact journal, authors had statistically significant decreases in research funding FCOI in the first 3 years postpublication compared to the year prior.
- Research Article
13
- 10.1186/s13643-020-01318-5
- Apr 8, 2020
- Systematic Reviews
BackgroundA previous study found that 2 of 29 (6.9%) meta-analyses published in high-impact journals in 2009 reported included drug trials’ funding sources, and none reported trial authors’ financial conflicts of interest (FCOIs) or industry employment. It is not known if reporting has improved since 2009. Our objectives were to (1) investigate the extent to which pharmaceutical industry funding and author-industry FCOIs and employment from included drug trials are reported in meta-analyses published in high-impact journals and (2) compare current reporting with results from 2009.MethodsWe searched PubMed (January 2017–October 2018) for systematic reviews with meta-analyses including ≥ 2 randomized controlled trials (RCTs) of patented drugs. We included 3 meta-analyses published January 2017–October 2018 from each of 4 high-impact general medicine journals, high-impact journals from 5 specialty areas, and the Cochrane Database of Systematic Reviews, as in the previous study.ResultsAmong 29 meta-analyses reviewed, 13 of 29 (44.8%) reported the funding source of included trials compared to 2 of 29 (6.9%) in 2009, a difference of 37.9% (95% confidence interval, 15.7 to 56.3%); this included 7 of 11 (63.6%) from general medicine journals, 3 of 15 (20.0%) from specialty medicine journals, and 3 of 3 (100%) Cochrane reviews. Only 2 of 29 meta-analyses (6.9%) reported trial author FCOIs, and none reported trial author-industry employment.Protocol PublicationA protocol was uploaded to the Open Science Framework prior to initiating the study. https://osf.io/8xt5p/LimitationsWe examined only a relatively small number of meta-analyses from selected high-impact journals and compared results to a similarly small sample from an earlier time period.ConclusionsReporting of drug trial sponsorship and author FCOIs in meta-analyses published in high-impact journals has increased since 2009 but is still suboptimal. Standards on reporting of trial funding described in the forthcoming revised PRISMA statement should be adapted and enforced by journals to improve reporting.
- Research Article
- 10.1056/wh200609070000004
- Jan 1, 2006
- NEJM Journal Watch
In the U.S., women constitute 49% of medical students and 32% of full-time medical faculty. To quantify the representation of the sexes in academic medical authorship, the investigators assessed original articles published in 1970, 1980, 1990, 2000, and 2004 in six journals that cover four core specialties and that are generally regarded as prestigious: Annals of Internal Medicine, Annals of Surgery, Journal of …
- Research Article
74
- 10.1016/j.jclinepi.2012.10.005
- Jan 22, 2013
- Journal of Clinical Epidemiology
Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis
- Front Matter
31
- 10.1053/j.gastro.2013.02.013
- Feb 21, 2013
- Gastroenterology
Bibliometrics of the Top 100 Clinical Articles in Digestive Disease
- Research Article
9
- 10.1186/s13613-021-00954-x
- Nov 27, 2021
- Annals of Intensive Care
BackgroundCritical care randomized controlled trials (RCTs) are often published in high-impact journals, whether general journals [the New England Journal of Medicine (NEJM), The Lancet, the Journal of the American Medical Association (JAMA)] or critical care journals [Intensive Care Medicine (ICM), the American Journal of Respiratory and Critical Care Medicine (AJRCCM), Critical Care Medicine (CCM)]. As rejection occurs in up to 97% of cases, it might be appropriate to assess pre-submission probability of being published. The objective of this study was to develop and internally validate a simplified score predicting whether an ongoing trial stands a chance of being published in high-impact general journals.MethodsA cohort of critical care RCTs published between 1999 and 2018 in the three highest impact medical journals (NEJM, The Lancet, JAMA) or the three highest impact critical care journals (ICM, AJRCCM, CCM) was split into two samples (derivation cohort, validation cohort) to develop and internally validate the simplified score. Primary outcome was journal of publication assessed as high-impact general journal (NEJM, The Lancet, JAMA) or critical care journal (ICM, AJRCCM, CCM).ResultsA total of 968 critical care RCTs were included in the predictive cohort and split into a derivation cohort (n = 510) and a validation cohort (n = 458). In the derivation cohort, the sample size (P value < 0.001), the number of centers involved (P value = 0.01), mortality as primary outcome (P value = 0.002) or a composite item including mortality as primary outcome (P value = 0.004), and topic [ventilation (P value < 0.001) or miscellaneous (P value < 0.001)] were independent factors predictive of publication in high-impact general journals, compared to high-impact critical care journals. The SCOTI score (Sample size, Centers, Outcome, Topic, and International score) was developed with an area under the ROC curve of 0.84 (95% Confidence Interval, 0.80–0.88) in validation by split sample.ConclusionsThe SCOTI score, developed and validated by split sample, accurately predicts the chances of a critical care RCT being published in high-impact general journals, compared to high-impact critical care journals.
- Research Article
395
- 10.1136/bmj.d6128
- Oct 25, 2011
- BMJ
Objectives To assess the prevalence of honorary and ghost authors in six leading general medical journals in 2008 and compare this with the prevalence reported by authors of articles published...
- Front Matter
16
- 10.1097/aln.0000000000003830
- May 27, 2021
- Anesthesiology
Authorship and Publication Matters: Credit and Credibility.
- Front Matter
1
- 10.1053/j.jvca.2019.01.035
- Jan 16, 2019
- Journal of Cardiothoracic and Vascular Anesthesia
From the Brontës to J.K. Rowling and Beyond – Have We Hit a Wall? The Status of Women Authors
- Research Article
86
- 10.1136/bmjopen-2017-020089
- Apr 1, 2018
- BMJ Open
ObjectiveThe present study aims to elucidate the state of gender equality in high-quality dermatological research by analysing the representation of female authorships from January 2008 to May 2017.DesignRetrospective, descriptive study.Setting113...
- Research Article
10
- 10.1016/j.euf.2022.04.009
- May 6, 2022
- European Urology Focus
A Seat at the Table: The Correlation Between Female Authorship and Urology Journal Editorial Board Membership
- Ask R Discovery
- Chat PDF
AI summaries and top papers from 250M+ research sources.