Citation-based Metrics Research Articles

Editorial diversity correlates with journal impact factor and author diversity in cardiothoracic surgery.

While there are no widely accepted diversity, equity, and inclusion (DEI) metrics for journals, geographic and sex diversity across a journal's editorial board may provide a surrogate measure of its commitment to DEI. We explored the association between journal quality and DEI metrics for cardiothoracic surgery (CTS) journals and investigated whether editorial diversity correlates with diversity across published articles. We collected the following data for 30 CTS journals: country of publication (categorized by income level), journal quality metrics (citation-based metrics, e.g., impact factor (IF) or H-index), and sex and geographic representation across editorial boards and published articles. Bivariate correlations between numeric variables were assessed using Spearman's correlation. Female representation across editorial boards was 12.1%. Most editorial board members belonged to the United States (35.2%), with only 7.4% from the lower-middle-income countries and 0% from low-income countries. IF showed a strong positive correlation with female editorial representation (r=0.70) but an inverse correlation with low- and middle-income countries (LMICs) editorial representation (r=-0.45). Female editorial representation demonstrated a significant positive correlation with female first authorship (r=0.45), whereas LMIC editorial representation correlated strongly with LMIC corresponding authorship (r=0.85). Women and researchers from LMICs are in the minority across editorial boards of CTS journals. However, a strong correlation between journal H-index and female editorial representation indicates that top-ranked journals are spearheading efforts to improve equitable sex-based and gender representations. Similar efforts are required to ensure more global geographic representation across editorial boards and top-ranked CTS journals are the best placed to lead by example.

Dimensions: Calculating disruption indices at scale

Abstract Assessing the disruptive nature of a line of research is a new area of academic evaluation that moves beyond standard citation-based metrics by taking into account the broader citation context of publications or patents. The “CD index” and a number of related indicators have been proposed in order to characterize the disruptiveness of scientific publications or patents. This research area has generated a lot of attention in recent years, yet there is no general consensus on the significance and reliability of disruption indices. More experimentation and evaluation would be desirable, but it is hampered by the fact that the calculation of these indicators is time-consuming, especially if done at scale on large citation networks. We present a novel SQL-based method to calculate disruption indices for the Dimensions publications data on Google BigQuery. This reduces the computational time taken to produce such indices by an order of magnitude, as well as making available such functionalities within an online environment that requires no setup efforts. We explain the novel algorithm and describe how its results align with preexisting implementations of disruption indicators. This method will enable researchers to develop, validate, and improve disruption models more quickly and with more precision.

Vascularised Composite Allotransplantation – A guide to optimal dissemination of scientific outputs

PurposeThe impact of academic research is not just important within the clinical domain but within society as a whole. Altmetric Attention Score (AAS) offers a means of assessing how scholarly outputs are interacted with online. Vascularised Composite Allotransplantation (VCA) is a modern but rapidly evolving topic which encompasses a broad range of complex and clinically significant surgical interventions. Primarily VCA is utilised in the reconstruction of complex, composite tissue defects, including limb and face transplantation. There is also a growing interest in the role of VCA as an early means of real-time immuno-monitoring, in sentinel skin flap transplant (SSF). Materials & methodsIn July 2024, a search was conducted using the Altmetric Explorer (AE) database using the search term ‘vascular-composite’ AND ‘allograft’ OR ‘allotransplant’. A simultaneous literature search was carried out using Web of Science (WoS) database utilising the same search terms with extraction of traditional citation-based metrics as well as relevant ‘meso’ and ‘micro’ subject headings. Corresponding citation-based metrics were extracted utilising SCImago. Data was compiled and analysed using a linear regression model with level of significance set at p < 0.05. ResultsThe Top 100 (T100) performing articles relating to VCA displayed a mean Altmetric Attention Score (AAS) of 3.31. All T100 papers were published in the English language. Sixty percent (n = 60) of T100 papers were published in Q1 Journals. News outlet mentions (r = 6.95), blog mentions (r = 6.20), and Twitter/X mentions (r = 0.52) demonstrated the greatest positive impact on AAS upon application of a linear regression model (p < 0.05). ConclusionAltmetric Scores offer a means of appraising the impact of research outputs in both academic and societal domains. Such modern metrics are useful in evolving topics such as VCA as AAS is not dependent on citation counts. Publishing of outputs in high quartile, open access journals with timely utilisation of news and social media outlets should be utilised by researchers aiming to maximise dissemination of research outputs in the field of VCA.

Social media for cardiac imagers: a review.

Cardiac imaging plays a pivotal role in the diagnosis and management of cardiovascular diseases. In the burgeoning landscape of digital technology and social media platforms, it becomes essential for cardiac imagers to know how to effectively increase the visibility and the impact of their activity. With the availability of social sites like X (formerly Twitter), Instagram and Facebook, cardiac imagers can now reach a wider audience and engage with peers, sharing their findings, insights, and discussions. The integration of persistent identifiers, such as Digital Object Identifiers (DOIs), facilitates traceability and citation of cardiac imaging publications across various digital platforms, further enhancing their discoverability. To maximize visibility, practical advice is provided, including the use of visually engaging infographics and videos, as well as the strategic implementation of relevant hashtags and keywords. These techniques can significantly improve the discoverability of cardiac imaging research through search engine optimization and social media algorithms. Tracking impact and engagement is crucial in the digital age, and this review discusses various metrics and tools to gauge the reach and influence of cardiac imaging publications. This includes traditional citation-based metrics and altmetrics. Moreover, this review underscores the importance of creating and updating professional profiles on social platforms and participating in relevant scientific communities online. The adoption of digital technology, social platforms, and a strategic approach to publication sharing can empower cardiac imaging professionals to enhance the visibility and impact of their research, ultimately advancing the field and improving patient care.

The Top 100 Most Impactful Articles on the Achilles Tendon According to Altmetric Attention Score and Number of Citations.

Achilles tendon injuries often generate substantial discussion in the mainstream media. The Altmetric Attention Score (AAS) measures the online dialogue pertaining to Achilles tendon research that occurs outside scientific journals, which traditional citation-based metrics fail to capture. To characterize the top 100 most-cited Achilles tendon articles and compare them with the 100 Achilles tendon articles with the highest AAS. A secondary goal was to gain an improved understanding of the online dissemination and interpretation of Achilles tendon research through this comparison. Cross-sectional study. The Web of Science Clarivate database was queried to isolate the 100 most-cited Achilles tendon articles, and the Altmetric database was queried to identify the Achilles tendon articles with the top 100 AAS values. Data elements were extracted for each article including study type, study topic, and geographic origin. The Web of Science Clarivate database search yielded 10,890 articles published between 1970 and 2021. The 100 most-cited articles were published in 35 journals, with the American Journal of Sports Medicine being the most prevalent. The mean (±SD) number of citations was 214.5 ± 86.47. The most prevalent study type was laboratory (28.0%). The most prevalent study topic was treatment (41.0%). Of these articles, 72.0% were European. The Altmetric database search yielded 3810 articles published between 1957 and 2021. The AAS of the top 100 articles ranged from 37 to 476 with a mean of 98.17 ± 85.53. The selected articles were published in 39 journals, with the British Journal of Sports Medicine being the most prevalent. The most prevalent study type was randomized controlled trial (25.0%). The most common study topic was treatment (40.0%). Of these articles, 46.0% were European. Our findings suggest that, although the scientific community remains committed to high-impact journals with articles backed by high citation numbers, there is an increasing opportunity to consume Achilles tendon literature through social media.

The extreme upper tail of Japan’s citation distribution reveals its research success

A number of indications, such as the number of Nobel Prize winners, show Japan to be a scientifically advanced country. However, standard bibliometric indicators place Japan as a scientifically developing country. The present study is based on the conjecture that Japan is an extreme case of a general pattern in highly industrialized countries. In these countries, scientific publications come from two types of studies: some pursue the advancement of science and produce highly cited publications, while others pursue incremental progress and their publications have a very low probability of being highly cited. Although these two categories of papers cannot be easily identified and separated, the scientific level of Japan can be tested by studying the extreme upper tail of the citation distribution of all scientific articles. In contrast to standard bibliometric indicators, which are calculated from the total number of papers or from sets of papers in which the two categories of papers are mixed, in the extreme upper tail, only papers that are addressed to the advance of science will be present. Based on the extreme upper tail, Japan belongs to the group of scientifically advanced countries and is significantly different from countries with a low scientific level. The number of Clarivate Citation laureates also supports our hypothesis that some citation-based metrics do not reveal the high scientific level of Japan. Our findings suggest that Japan is an extreme case of inaccuracy of some citation metrics; the same drawback might affect other countries, although to a lesser degree.

Impact of medical technologies may be predicted using constructed graph bibliometrics

Scientific research is driven by allocation of funding to different research projects based in part on the predicted scientific impact of the work. Data-driven algorithms can inform decision-making of scarce funding resources by identifying likely high-impact studies using bibliometrics. Compared to standardized citation-based metrics alone, we utilize a machine learning pipeline that analyzes high-dimensional relationships among a range of bibliometric features to improve the accuracy of predicting high-impact research. Random forest classification models were trained using 28 bibliometric features calculated from a dataset of 1,485,958 publications in medicine to retrospectively predict whether a publication would become high-impact. For each random forest model, the balanced accuracy score was above 0.95 and the area under the receiver operating characteristic curve was above 0.99. The high performance of high impact research prediction using our proposed models show that machine learning technologies are promising algorithms that can support funding decision-making for medical research.

Research impact in randomized controlled trials of diabetes: an altmetric approach.

This study aimed to assess the impact of research in randomized controlled trials (RCTs) of diabetes and explore the various subject areas related to diabetes that receive attention on social media platforms. Altmetric measures were utilized to collect and extract relevant data, providing valuable insights into the social reach and influence of clinical research beyond traditional citation-based metrics. The research focused on RCTs of diabetes involving at least one Iranian author, indexed in Scopus. Altmetric.com was employed to extract altmetric data, and the collected articles were categorized into 14 subject areas for individual analysis using STATA. The analysis revealed that a majority of the diabetes studies examined nutrition, complications, treatment, genetics, basic mechanisms, and comorbidities of the disease. Conversely, subject areas such as diagnosis, education, gestational diabetes, psychology, physical activity, prevention, dentistry, and economics had fewer studies associated with them. Among social media platforms, Twitter, Facebook, Google+, and Reddit emerged as the most frequently mentioned platforms. Furthermore, Mendeley readership was identified as the preferred platform for engagement across several subject areas. The substantial number of social media mentions indicates a significant level of public interest and concern regarding diabetes. Social media platforms serve as effective tools for disseminating research findings from clinical trials. Altmetric data proves valuable to researchers and funding agencies seeking to comprehend the impact of their work, enabling them to allocate resources more effectively.

Citation beneficiaries of discipline-specific mega-journals: who and how much

The emergence of mega-journals (MJs) has influenced scholarly communication. One concrete manifestation of this impact is that more citations have been generated. Citations are the foundation of many evaluation metrics to assess the scientific impact of journals, disciplines, and regions. We focused on searching for citation beneficiaries and quantifying the relative benefit at the journal, discipline and region levels. More specifically, we examined the distribution and contribution to citation-based metrics of citations generated by the five discipline-specific mega-journals (DSMJs) categorized as Environmental Sciences (ES) on Web of Science (WoS) from Clarivate Analytics in 2021: Sustainability, International Journal of Environmental Research and Public Health, Environmental Science and Pollution Research, Journal of Cleaner Production and Science of the Total Environment. Analysis of the distribution of citing data of the five DSMJs shows a pattern with wide coverage but skewness by region and the WoS category; that is, papers in the five DSMJs contributed 26.66% of their citations in 2021 to Mainland China and 22.48% to the ES. Moreover, 15 journals within the ES had their JIFs boosted by more than 20%, benefitting from the high citing rates of the five DSMJs. More importantly, the analysis provides clear evidence that DSMJs can contribute to JIF scores throughout a discipline through their volume of references. Overall, DSMJs can widely impact scholarly evaluation because they contribute citation benefits and improve the evaluation index performance of different scientific entities at different levels. Considering the important application of citation indicators in the academic evaluation system and the increase in citations, it is important to reconsider the real research impact that citations can reflect.

Beyond views, productivity, and citations: measuring geopolitical differences of scientific impact in communication research

Scientometric analyses applying critical sociological frameworks have previously shown that high-prestige research output—with regards to both quantity and impact—is typically clustered in a few core countries and world regions, indicating uneven power relations and systematic biases within global academia. Although citation count is a common formula in these analyses, only a handful of studies investigated altmetrics (impact measures beyond citation-based metrics) in communication science. In this paper, we explore geopolitical biases of impact amongst the most productive scholars in the field of communication from 11 countries and 3 world regions. Drawing on SCOPUS data, we test three formulas that measure scholarly performance (citations per document; views per document; and citations per view) to investigate how geographical location affects the impact of scholars. Our results indicate a strong US-dominance with regard to citation-based impact, emphasizing a further need for de-Westernization within the field. Moreover, the analysis of altmetric formulas revealed that research published by Eastern European and Spanish scholars, although accessed similarly or even more often than American or Western European publications, is less cited than those. Country-level comparisons are also discussed.

Identification and Portraits of Open Access Journals Based on Open Impact Metrics Extracted from Social Activities

This article focuses on open impact metrics extracted from social media activities that demonstrate the identification and portraits of open access journals based on these alternative forms of open impact metrics. The research sample consists of open access journals from Scopus, with open impact metrics retrieved from Altmetric.com . The open impact metrics extracted from social activities established that an evaluation system based on altmetrics can better reflect the portraits of open access journals than traditional citation-based metrics. This study finds that open access journals strengthen international academic communication and cooperation, build cross-border and cross-regional knowledge-sharing projects, realize the knowledge of interdisciplinary sharing and exchange, and, most importantly, provide a one-stop service for readers. This research indicates that through the use of open impact metrics, it is possible to identify the portraits of open access journals, thus providing a new method to construct and reform open access journal evaluation systems.

The Online Attention to Research in Legal Medicine: An Altmetric Study on the Most Discussed Articles on the Web

Background: Online attention is a relevant research feature that can be related to its societal impact. Online attention in the field of legal medicine was explored as well as the relation between citation count and the Altmetric Attention Score (AAS). Methods: A search was performed, through the free Dimensions app, during September 2020, on the journals related to legal medicine listed in the 2019 edition of the Journal Citation Reports (JCR). The 200 published items with the highest AAS were retrieved and screened for bibliometric data. Articles with related citations were gathered from Web of Science (WOS), Scopus, and Dimensions. Results: Seventy-five percent of the articles were published by five of the selected journals. Twitter and news outlets were the most prevalent Altmetric resources. Additionally, 73.5% of the articles were published between 2013 and 2020. The most frequent topics were the ones related to toxicology and pharmacology, followed by criminalistics and law and bioethics. A poor correlation was found between the citations in WOS, Dimensions, and Scopus and the AAS. Conclusions: The online attention on legal medicine-related topics can be considered high on social media. The high number of news outlets is a distinctive feature of the most discussed articles in legal medicine. In this field of medicine, citation-based metrics combined with Altmetric can allow a broader evaluation of research findings but it should be underlined that, when measuring very different aspects of a published item, the scores in classic citation databases and Altmetric do not have a direct relation.

Do linguistic features of research article titles affect received online attention? A corpus-based analysis

Purpose Scientific impact is traditionally assessed with citation-based metrics. Recently, altmetric indices have been introduced to measure scientific impact both within academia and among the general public. However, little research has investigated the association between the linguistic features of research article titles and received online attention. To address this issue, the authors examined in the present study the relationship between a series of title features and altmetric attention scores.Design/methodology/approach The data included 8,658 titles of Science articles. The authors extracted six features from the title corpus (i.e. mean word length, lexical sophistication, lexical density, title length, syntactic dependency length and sentiment score). The authors performed Spearman’s rank analyses to analyze the correlations between these features and online impact. The authors then conducted a stepwise backward multiple regression to identify predictors for the articles' online impact.Findings The correlation analyses revealed weak but significant correlations between all six title features and the altmetric attention scores. The regression analysis showed that four linguistic features of titles (mean word length, lexical sophistication, title length and sentiment score) have modest predictive effects on the online impact of research articles.Originality/value In the internet era with the widespread use of social media and online platforms, it is becoming increasingly important for researchers to adapt to the changing context of research evaluation. This study identifies several linguistic features that deserve scholars’ attention in the writing of article titles. It also has practical implications for academic administrators and pedagogical implications for instructors of academic writing courses.

Analyzing the Relationship between Citation-Based Impact Metrics and Electronic Journal Usage: A Case Study

This study examines the relationship between Journal Impact Factors (JIF) and e-journal usage in collection management. We focus on the impact of major JIFs on local e-journal usage and propose an alternative approach to conventional methods for collection selectors. By treating journal usage patterns as panel data and employing fixed-effects regression models, we find that journal popularity has the greatest influence on local e-journal usage and the effects of impact factors on academic article usage can vary across different disciplines. The methodology and findings of our study offer valuable insights and practical implications for academic librarians overseeing collection development.

Beyond Citation Counts: Reassessing Top Criminologists’ “Influence” With Altmetric Scores

Criminal justice and criminology (CCJ), like many academic disciplines, conducts its share of rankings. Citation-based ranks of individual scholars are particularly popular, and they tend to consistently identify the field’s supposedly “top” scholars and “academic stars.” Whether citations equate with “influence,” however, is up for debate. At the least, citation-based metrics are unidimensional and fail to capture attention outside academia. Accordingly, we drew on the work of Cohn et al. and re-ranked top-cited scholars using the Google Chrome “Altmetric it!” bookmarklet. As expected, the Altmetrics methodology fundamentally altered past rankings. The most influential scholars in our rankings, Terrie E. Moffitt and Avshalom Caspi, received higher Altmetric scores than all the remaining ranked scholars combined.

Social Media Engagement for Urology Journals — A Correlation Analysis of Traditional and Social Media Metrics

IntroductionThe growing adoption of social media (SoMe) by the scientific community has cemented the role of SoMe in information dissemination and engagement of academic work. The objective of this study is to evaluate the relationship between traditional and alternative SoMe metrics of urology journals.MethodsUrology journals listed on the SCImago Journal &amp; Country Rank (SJR) electronic portal were selected and data pertaining to traditional metrics were collected. Official SoMe platforms of eligible journals were identified and indicators of online activity were recorded. Correlations between traditional metrics (SJR, h-index, and Scopus CiteScore) and social metrics were performed via Spearman rank-order correlation.ResultsOf 107 journals, 54.2% of journals had at least one form of SoMe presence. The median SJR (0.535 versus 0.334, P = 0.005), h-index (34 versus 20, P = 0.001), and Scopus CiteScore (3.25 versus 2.20, P = 0.014) were significantly higher among journals with SoMe networks. All 3 traditional indicators demonstrated strong global correlations with various Twitter-based metrics (rs = 0.428 to 0.571). In particular, SoMe journals with more than 3000 citations in the previous 3 years also displayed very strong correlations between all 3 traditional metrics and alternative social metrics (rs = 0.714 to 0.821).ConclusionsJournals with SoMe presence had significantly higher traditional metric values (SJR, h-index, and CiteScore) compared to journals without SoMe presence. Strong, positive correlations between citation-based and alternative social metrics were also observed. Alternative social metrics may be harnessed as supplemental indicators of a journal’s scientific impact.

Top 50 most-cited articles on SMILE surgery between 2010 and 2022: a correlation comparison between conventional bibliometrics and current altmetrics of research impact.

To conduct a bibliometric and altmetric analysis of the top 50 most-cited articles on small incision lenticule extraction (SMILE) surgery and assess its correlations with other metrics. The terms "small incision lenticule extraction" or "SMILE" were searched in the title, abstract, and keywords using Web of Science database. The retrieved articles (n = 927, between 2010 and 2022) were analyzed in-depth using altmetric attention scores (AAS), and traditional metrics (citation numbers of articles, impact factor of journals, and other citation-based metrics). A correlation statistics was performed with metrics. The articles' focus was examined quantitatively and the most prolific parameters were identified. Authorship network and country statistics also was examined. Citation numbers were between 491 and 45. AASs were between 26 and 0. The altmetric score correlated moderately with citation number (r = 0.44, P = 0.001) and annual mean citations (r = 0.49, P < 0.001), but correlated weakly with impact factor (r = 0.28, P = 0.045), and immediacy index (r = 0.32, P = 0.022). The most articles published from China and the most articles were published in 2014. Modern SMILE surgery was mostly compared to the older LASIK procedure. The highest authorship number of links belonged to Zhou XT. The first bibliometric and altmetric analysis of SMILE research provides novel directions for future work by showing the current research trends, prolific parameters, and zones with potential for the public's attention, providing useful information about the dissemination of scientific knowledge on SMILE in social media and to the general public.

Deep representation learning of scientific paper reveals its potential scholarly impact

Citation and citation-based metrics are traditionally used to quantify the scholarly impact of scientific papers. However, for documents without citation data, i.e., newly published papers, the citation-based metrics are not available. By leveraging deep representation techniques, we propose a text-content based approach that may reveal the scholarly impact of papers without human domain-specific knowledge. Specifically, a large-scale Pre-Trained Model (PTM) with 110 million parameters is utilized to automatically encode the paper into the vector representation. Two indicators, τ (Topicality) and σ (Originality), are then proposed based on the learned representations. These two indicators leverage the spatial relations of paper representations in the semantic space to capture the impact-related characteristics of a scientific paper. Extensive experiments have been conducted on a COVID-19 open research dataset with 1,056,660 papers. The experimental results demonstrate that the deep representation learning method can better capture the scientific content in the published literature; and the proposed indicators are positively and significantly associated with a paper’s potential scholarly impact. In the multivariate regression analysis for the potential impact of a paper, the coefficients of σ and τ are 5.4915 (P<0.001) and 6.6879 (P<0.001) for next 6 months prediction, 12.9964 (P<0.001) and 13.8678 (P<0.001) for next 12 months prediction. The proposed framework may facilitate the study of how scholarly impact is generated, from a textual representation perspective.

An apologia for using journal prestige in academic employment decisions: An asymmetric information perspective

Using journal prestige to gauge the quality of research articles has been criticized by some scholars in the academic research assessment literature. These critics argue that citation-based metrics, while imperfect, are better suited to assessing article quality. This paper counters these beliefs by describing specific research assessment contexts where journal prestige is likely to be the superior mode of assessment. To do so, academic employment decisions are framed as asymmetric information problems wherein the research evaluating body has less information about a scholar’s true abilities than the scholar. Signaling Theory therefore offers a logical explanation for using journal prestige as a measure of research quality during academic contract renewals and promotions wherein recent research accomplishments are of preeminent importance. While journal prestige is an imperfect measure of article quality (as are citation counts), understanding it as a credible signal in an asymmetric information environment proffers a fundamental explanation for its applicability in academic assessment decisions.

Do open citations give insights on the qualitative peer-review evaluation in research assessments? An analysis of the Italian National Scientific Qualification

In the past, several works have investigated ways for combining quantitative and qualitative methods in research assessment exercises. Indeed, the Italian National Scientific Qualification (NSQ), i.e. the national assessment exercise which aims at deciding whether a scholar can apply to professorial academic positions as Associate Professor and Full Professor, adopts a quantitative and qualitative evaluation process: it makes use of bibliometrics followed by a peer-review process of candidates’ CVs. The NSQ divides academic disciplines into two categories, i.e. citation-based disciplines (CDs) and non-citation-based disciplines (NDs), a division that affects the metrics used for assessing the candidates of that discipline in the first part of the process, which is based on bibliometrics. In this work, we aim at exploring whether citation-based metrics, calculated only considering open bibliographic and citation data, can support the human peer-review of NDs and yield insights on how it is conducted. To understand if and what citation-based (and, possibly, other) metrics provide relevant information, we created a series of machine learning models to replicate the decisions of the NSQ committees. As one of the main outcomes of our study, we noticed that the strength of the citational relationship between the candidate and the commission in charge of assessing his/her CV seems to play a role in the peer-review phase of the NSQ of NDs.