Future Citations Research Articles

Referee quality and article citations: a case study of a finance journal

ABSTRACT As a result of the open science movement, we take advantage of one of the few journals in business studies, the Journal of Financial Economics (JFE), which between 2009 and 2021 named some of their referees. This allows us to produce a novel hand-collected data set to explore the extent to which referee quality matters in identifying articles that will receive strong future citations. We find that prominent academics are more likely to identify articles that will exhibit a high number of citations. This relationship is most pronounced among referees who are in a relatively mature stage in their careers. These results indicate that high-quality researchers, especially those in a relatively mature stage, are more likely to identify the potential of an article. To maximize future citations, editors thus need to consider the research publication record of the referees when making their selections. Overall, we provide insights which are of interest and relevance across higher education research.

Which orthodontic articles are accessed online the most? Exploring article usage metrics along with citations and altmetrics.

To investigate the current state of article usage metrics in orthodontics. Out of all orthodontic journals listed in Journal Citation Reports 2022, the European Journal of Orthodontics and The Angle Orthodontist fulfilled the inclusion criteria. All journal issues published in 2021 were scrutinized for original research articles and systematic reviews/meta-analyses, and the following features were collected: article type, subject, title, number of words and authors, Altmetric Attention Score (AAS), X (formerly Twitter) posts, and Mendeley reads. Article citations, number of publications, and h-index of the last authors were retrieved from Web of Science (WoS). 181 articles were considered eligible. The median number of views and downloads of included articles was 1296 (range: 355-10,233) and 793 (range: 167-3629). Page views, downloads, and total views were significantly correlated with WoS citations (rho > 0.345; P < .001). There was no correlation between usage metrics, AAS, X posts, and Mendeley reads. Number of downloads were significantly higher in studies dealing with new technologies, and where the last author had 1-40 publications or an h-index of 0-30. Page views, downloads, and total views were positively correlated with WoS citations and, therefore, may serve as an early estimate of future citations. Significant variations in article downloads may be expected in relation to article subject, scientific productivity, and impact of the last authors.

Ten year citation prediction model for systematic reviews using early years citation data

Citation counts are frequently used for assessing the scientific impact of articles. Current approaches for forecasting future citations counts have important limitations. This study aims to analyse and predict the trajectories of citation counts of systematic reviews (SR) based on their citation profiles in the previous years and predict quantiles of future citation counts. We included all SR published between 2010 and 2012 in medical journals indexed in the Web of Science. A longitudinal k-means (KML) clustering approach was applied to identify trajectories of citations counts 10 years after publication, according to the yearly citation count, the proportion of all cites attained in a specific year and the annual variation in citation counts. Finally, we built multinomial logistic regression models aiming to predict in what tercile or quartile of citation counts a SR would be 10 years after publication. Using clustering approaches, we obtained 24 groups of SR. Two groups (7.9% of the articles) had an average of > 200 citations, while two other groups (10.4% of the articles) presented an average of < 10 citations. The model predicting terciles of citation counts attained an accuracy of 72.8% (95%CI = 71.1–74.3%) and a kappa coefficient of 0.59 (95%CI = 0.57–0.62). Prediction of citation quartiles (combining the second and third quartiles into a single group) attained a accuracy of 76.2% (95%CI = 74.7–77.8%) and a kappa coefficient of 0.62 (95%CI = 0.59–0.64). This study provides an approach for predicting of future citations of SR based exclusively on citation counts from the previous years, with the models developed displaying an encouraging accuracy and agreement.

FutureCite: Predicting Research Articles’ Impact Using Machine Learning and Text and Graph Mining Techniques

The growth in academic and scientific publications has increased very rapidly. Researchers must choose a representative and significant literature for their research, which has become challenging worldwide. Usually, the paper citation number indicates this paper’s potential influence and importance. However, this standard metric of citation numbers is not suitable to assess the popularity and significance of recently published papers. To address this challenge, this study presents an effective prediction method called FutureCite to predict the future citation level of research articles. FutureCite integrates machine learning with text and graph mining techniques, leveraging their abilities in classification, datasets in-depth analysis, and feature extraction. FutureCite aims to predict future citation levels of research articles applying a multilabel classification approach. FutureCite can extract significant semantic features and capture the interconnection relationships found in scientific articles during feature extraction using textual content, citation networks, and metadata as feature resources. This study’s objective is to contribute to the advancement of effective approaches impacting the citation counts in scientific publications by enhancing the precision of future citations. We conducted several experiments using a comprehensive publication dataset to evaluate our method and determine the impact of using a variety of machine learning algorithms. FutureCite demonstrated its robustness and efficiency and showed promising results based on different evaluation metrics. Using the FutureCite model has significant implications for improving the researchers’ ability to determine targeted literature for their research and better understand the potential impact of research publications.

Forecasting Future Research Trends in the Construction Engineering and Management Domain Using Machine Learning and Social Network Analysis

Construction Engineering and Management (CEM) is a broad domain with publications covering interrelated subdisciplines and considered a key source of knowledge sharing. Previous studies used scientometric methods to assess the current impact of CEM publications; however, there is a need to predict future citations of CEM publications to identify the expected high-impact trends in the future and guide new research efforts. To tackle this gap in the literature, the authors conducted a study using Machine Learning (ML) algorithms and Social Network Analysis (SNA) to predict CEM-related citation metrics. Using a dataset of 93,868 publications, the authors trained and tested two machine learning classification algorithms: Random Forest and XGBoost. Validation of the RF and XGBoost resulted in a balanced accuracy of 79.1% and 79.5%, respectively. Accordingly, XGBoost was selected. Testing of the XGBoost model revealed a balanced accuracy of 80.71%. Using SNA, it was found that while the top CEM subdisciplines in terms of the number of predicted impactful papers are “Project planning and design”, “Organizational issues”, and “Information technologies, robotics, and automation”; the lowest was “Legal and contractual issues”. This paper contributes to the body of knowledge by studying the citation level, strength, and interconnectivity between CEM subdisciplines as well as identifying areas more likely to result in highly cited publications.

Shaping the Smart Search Possible: An Inside Story of ETD Metadata

Metadata is very important in scientific literature. Metadata in electronic dissertations and dissertations (ETDs) requires a modular workflow based on a flexible architecture. Metadata methods must be simplified by adapting procedures for different document formats and structures. The structure of the dissertation document, including Fig.s, tables, footnotes, etc., requires greater attention to the identification and organization of metadata in the ETD for rich textual content. Reliable extraction of appropriate metadata elements, such as title, author, abstract, keywords, etc., is essential for future citation capabilities. Such mechanisms are crucial for evaluating processes and adjusting ETD metadata to facilitate architecture expansion. Most modern ETDs rely on the organization of quality text documents, ranging from raw PDF documents or semi-structured XML documents to uniform standard TEI (Text Encoding Initiative) documents. The present study discusses metadata creation and harvesting, various extraction methods and digital library search, metadata for MARC records in ETD, and access to ETD metadata in the context of the current scenario. It also highlights the tags adopted for the creation of the MARC records in JNU Central Library ETD System for simple-level learning

Letter from Seth Barrett Tillman to the Honorable Clarence Thomas on Suggested Sources for a Future Citation

Letter from Seth Barrett Tillman to the Honorable Clarence Thomas on Suggested Sources for a Future Citation

Publication Rates of Abstracts Presented at American Orthopaedic Society for Sports Medicine Annual Meetings From 2016 to 2019.

Research in the form of poster and podium abstracts is disseminated at subspecialty society meetings. The quality of this research can be defined by exploring the ultimate publication rate of the presented abstracts. To investigate (1) the manuscript publication rate of abstracts presented at the American Orthopaedic Society for Sports Medicine (AOSSM) annual meeting; (2) whether abstract format (poster vs podium) influences overall or 2-year publication rates and time to publication; (3) the abstract factors that are associated with increased publication rate; and (4) whether publication quality as measured by journal of publication, level of evidence (LOE), and number of citations differs between posters and podiums. Cross-sectional study. Poster and podium abstracts that were presented at the AOSSM annual meetings between January 1, 2016, and December 31, 2019, were included. The PubMed and GoogleScholar databases were searched by abstract title and authors to determine whether the related manuscript had been published. For published manuscripts, the journal, journal impact factor (IF), time to publication, authors, and LOE were recorded. The manuscripts of 664 abstracts (341 poster, 323 podium presentations) were published during the study period. The overall publication rate was 52.4%. Publication within 2 years of the meeting was found to be higher in podium abstracts (45.8%) compared with poster abstracts (37.8%) (P = .0366). Podium abstracts had a shorter time to publication (P < .001), higher LOE (P = .0166), more citations (P < .0001), and were published in higher IF journals (P = .0028). Poster presentations were more likely to undergo a change in first author between the time of the conference and future publication (P = .0300). The most common journal of publication was the American Journal of Sports Medicine (36.8%). Abstracts presented at the AOSSM annual meeting had a high rate of publication within 2 years. There was no difference in publication rates between podium and poster abstracts, but podium abstracts had a shorter time to publication and more future citations and were published in journals with higher IFs.

A research landscape bibliometric analysis on climate change for last decades: Evidence from applications of machine learning

Climate change (CC) is one of the greatest threats to human health, safety, and the environment. Given its current and future impacts, numerous studies have employed computational tools (e.g., machine learning, ML) to understand, mitigate, and adapt to CC. Therefore, this paper seeks to comprehensively analyze the research/publications landscape on the MLCC research based on published documents from Scopus. The high productivity and research impact of MLCC has produced highly cited works categorized as science, technology, and engineering to the arts, humanities, and social sciences. The most prolific author is Shamsuddin Shahid (based at Universiti Teknologi Malaysia), whereas the Chinese Academy of Sciences is the most productive affiliation on MLCC research. The most influential countries are the United States and China, which is attributed to the funding activities of the National Science Foundation and the National Natural Science Foundation of China (NSFC), respectively. Collaboration through co-authorship in high-impact journals such as Remote Sensing was also identified as an important factor in the high rate of productivity among the most active stakeholders researching MLCC topics worldwide. Keyword co-occurrence analysis identified four major research hotspots/themes on MLCC research that describe the ML techniques, potential risky sectors, remote sensing, and sustainable development dynamics of CC. In conclusion, the paper finds that MLCC research has a significant socio-economic, environmental, and research impact, which points to increased discoveries, publications, and citations in the near future.

Philanthropic supported innovation: trends, areas, and impact

Philanthropy provides substantial support for science and research—does it do the same for technology and invention? We match patenting outcomes in the United States with philanthropic organizations as listed in the Candid database and find that at least 0.2% of U.S. patenting since 1976 received philanthropic support. After matching philanthropic patents to non-philanthropic patents based on technology, time, and the number of inventors, we find that philanthropic invention is more likely to be done by smaller teams and U.S. based inventors and less likely to rely on science research and non-U.S. prior art. Renewal rates are not significantly different from non-philanthropic patents and philanthropic patents are not more likely to be in underserved areas of technology. Philanthropic supported patents receive significantly fewer future prior art citations, thus implying that philanthropy supported patents are not exceptionally valuable.

Do the paper’s connections to existing work disclose its citation impact? A study based on graph representation learning

Influential scientific papers tend to be primarily based on combinations of prior works. However, assessing the potential impact of a new scientific paper remains a challenging task. In this article, we introduce an innovative framework to investigate the relationship between the embedding of citation networks and a paper’s future citation counts, based on the graph representation learning approach. First, we employ three Nobel Prize-winning topic papers from the Web of Science as our data source. Through data preprocessing and direct citation network modelling, we train the struc2vec model to obtain embeddings of papers’ citation network structure. Then, we perform visualisation and analysis on two types of networks. One is the direct-citation network, in which we identify four patterns of linkage between newly published papers and existing knowledge, and the other is the co-citation network, where we measure three structural variation indicators of new papers based on existing research findings. Finally, a statistical test is used to examine the predictive potentials of network embeddings. The results demonstrate that the structural features captured by the graph representation learning model can be used to predict a paper’s citation counts and impact. This article innovatively combines cluster analysis, visual analysis and statistical analysis to gain insights into the relationship between the hard-to-explain structural embeddings of newly published papers in a citation network and their future citations.

Competition and Innovation in the Financial Sector: Evidence from the Rise of FinTech Start-ups

This paper provides new evidence on the effects of entry on incumbents’ incentives to innovate by examining the rise of FinTech innovations over the period 2000-2016. We employ machine learning algorithms to classify a large sample of patent applications into five types of FinTech innovations. We then show that greater competition from innovators outside the financial sector increases the probability that incumbent financial firms will innovate. Our identification strategy exploits the variation over time in the share of FinTech patent applications by non-financial start-ups relative to incumbent financial firms, as a proxy for competitive pressures from outside the financial industry. We also find that this increased competition results in a higher number of FinTech patent applications by financial incumbents relative to non-financial ones, especially when the FinTech innovations are more important, as proxied by the number of their future patent citations.

Towards popularity prediction of information cascades via degree distribution and deep neural networks

Understanding paper citation dynamics and accurately predicting future citation counts of papers is of significant interest, and thus modeling citation dynamics as an information cascade has recently attracted considerable attention. Nevertheless, most of these recent deep learning-based information cascade prediction models are focused on the embedding of each individual node rather than the entire structure of the cascade graph, which limits the robustness of the model. Thus, instead of learning the representation of each node in the cascade, we propose learning the dynamic structural representation of the entire information cascade graph with the degree distribution vectors corresponding to different timestamps as the input of a sequential deep neural network, named CasDENN. Extensive experiments on datasets from academic paper citations (APS) and social media post forwards (Weibo) show a dramatic improvement over state-of-the-art baselines, where the prediction error can be reduced by approximately 8%–10% and the running time is less than 10% of the fast baseline.

Predicting substantive biomedical citations without full text

Insights from biomedical citation networks can be used to identify promising avenues for accelerating research and its downstream bench-to-bedside translation. Citation analysis generally assumes that each citation documents substantive knowledge transfer that informed the conception, design, or execution of the main experiments. Citations may exist for other reasons. In this paper, we take advantage of late-stage citations added during peer review because these are less likely to represent substantive knowledge flow. Using a large, comprehensive feature set of open access data, we train a predictive model to identify late-stage citations. The model relies only on the title, abstract, and citations to previous articles but not the full-text or future citations patterns, making it suitable for publications as soon as they are released, or those behind a paywall (the vast majority). We find that high prediction scores identify late-stage citations that were likely added during the peer review process as well as those more likely to be rhetorical, such as journal self-citations added during review. Our model conversely gives low prediction scores to early-stage citations and citation classes that are known to represent substantive knowledge transfer. Using this model, we find that US federally funded biomedical research publications represent 30% of the predicted early-stage (and more likely to be substantive) knowledge transfer from basic studies to clinical research, even though these comprise only 10% of the literature. This is a threefold overrepresentation in this important type of knowledge flow.

Do Articles Shared by Academic Medicine Social Media Influencers Drive Future Citation Rates?

To assess the role of influential figures within social media (SoMe) in driving future citations. All original articles published in the Journal of Urology (JU) and European Urology (EU) in 2018 were identified. For each article, number of mentions on any SoMe platform, article's Twitter reach, and total citations were collected. Article characteristics such as type of study, article topic, and open access status were identified. Total academic research output was obtained for first and last authors of included articles. Influential SoMe figures were defined as users that tweeted about included articles and had over 2000 followers. For these accounts, we collected total followers, total tweets, engagement statistics, verification status, and academic characteristics such as total citations and total prior publications. The impact of social media, article, and academic characteristics on future citations was assessed using panel data regression analysis. We identified 394 articles with 8,895 total citations and 460 SoMe influencers. On panel data regression modeling, tweets about a specific article were associated with future citations (0.17 citations per tweet about an article, p<0.001). SoMe influencer characteristics were not associated with increased citations (p>0.05). The following non-SoMe-associated characteristics were predictive of future citations (p<0.001): study type (prospective studies received 12.9 more citations than cross-sectional studies), open access status (4.3 citations more if open access, p<0.001), and previously well-published first and last authors. While SoMe posts are associated with increased visibility and higher future citation rates, SoMe influencers do not appear to drive these outcomes. Instead, high quality and accessibility were more predictive of future citability.

Article impact of the Financial Times 50 management journals

AbstractAn academic journal is produced by a voluntary organization of scholars who create bodies of common knowledge consisting of the journal's published articles, along with all the references that link to the sources of knowledge for those articles. Even journals in the same discipline develop distinct bodies of common knowledge. New articles simultaneously build on and contribute to this common knowledge and we argue that this interaction with a journal's common knowledge influences how an article will be cited in the future. We explore the ecosystem identified by the Financial Times 50 management journals, demonstrating the overlap and distinction in their common knowledge. We find that journals fundamentally differ in the body of knowledge that comprises them, the sources of knowledge that contribute to them, and the journals to which they make an impact. Our findings generate novel insights into how the references cited in an article affect future citations.

Measuring altmetric events: the need for longer observation period and article level computations

Purpose This paper aims to explore the impact of the data observation period on the computation of altmetric measures like velocity index (VI) and half-life. Furthermore, it also attempts to determine whether article-level computations are better than computations on the whole of the data for computing such measures. Design/methodology/approach The complete publication records for the year 2016 indexed in Web of Science and their altmetric data (original tweets) obtained from PlumX are obtained and analysed. The creation date of articles is taken from Crossref. Two time-dependent variables, namely, half-life and VI are computed. The altmetric measures are computed for all articles at different observation points, and by using whole group as well as article-level averaging. Findings The results show that use of longer observation period significantly changes the values of different altmetric measures computed. Furthermore, use of article-level delineation is advocated for computing different measures for a more accurate representation of the true values for the article distribution. Research limitations/implications The analytical results show that using different observation periods change the measured values of the time-related altmetric measures. It is suggested that longer observation period should be used for appropriate measurement of altmetric measures. Furthermore, the use of article-level delineation for computing the measures is advocated as a more accurate method to capture the true values of such measures. Practical implications The research work suggests that altmetric mentions accrue for a longer period than the commonly believed short life span and therefore the altmetric measurements should not be limited to observation of early accrued data only. Social implications The present study indicates that use of altmetric measures for research evaluation or other purposes should be based on data for a longer observation period and article-level delineation may be preferred. It contradicts the common belief that tweet accumulation about scholarly articles decay quickly. Originality/value Several studies have shown that altmetric data correlate well with citations and hence early altmetric counts can be used to predict future citations. Inspired by these findings, majority of such monitoring and measuring exercises have focused mainly on capturing immediate altmetric event data for articles just after the publication of the paper. This paper demonstrates the impact of the observation period and article-level aggregation on such computations and suggests to use a longer observation period and article-level delineation. To the best of the authors’ knowledge, this is the first such study of its kind and presents novel findings.

What do researchers cite in their Literature Review sections? An exploratory study of citations in Information Systems research articles

The Literature Review (LR) section is an integral part of a research article (RA) where the author needs to develop a theoretical groundwork to anchor his/her own study. However, the part-genre always eludes novice writers, which involves deciding what to cite from relevant past literature and what goals the cited knowledge serves therein. Insights are thus necessary into how experienced/expert writers address such issues. Unfortunately, there has been a dearth of research into this topic, with existing work focusing mainly on the formal features and generic functions of source use. Motivated by the missing gap and the pedagogical need and grounded in Swales’s (1990) CARS model and Authors’ (2014) semantic typology of citations, the present study is an attempt to examine the semantic attributes of source ideas that get cited in different parts of the LR sections of Information Systems RAs following a behavioral science research (BSR) paradigm. Findings reveal that distinct types of source ideas are cited, which are associated with the specific move-steps of the LR sections and the BSR paradigm. A comparison of some of the cited ideas with the original ideas from the source texts also demonstrates that the latter have been re-contextualized to varying degrees to advance the arguments that the writers make in their LRs. Taken together, the study’s outcomes suggest that using a move-specific approach to citations can yield useful insights into source use in LRs. Implications for citation teaching and future citation research will be drawn.

Exploring the “awakening effect” in knowledge diffusion: a case study of publications in the library and information science domain

• Uncovered the general mechanisms that may not only be held responsible for the awakening of SBs but also the attention received by papers. • Proved that the awakening effect does exist ubiquitously, and tried to quantify it. • Proposed a super-network model that may be used to depict the knowledge diffusion trajectory and a systematic feature-searching method that can find complicated features efficiently. Inspired by “sleeping beauties in science”, we proposed that the awakening effect in knowledge diffusion is ubiquitous, whereas the “prince” paper has the strongest effect. To test this hypothesis, a three-layer super-network model depicting the knowledge diffusion trajectory is designed and the diffusion path of the awakening effect (defined on the basis of influential strength) is simulated. In detail, the model is built based on the citation network and collaboration network of 63785 publications in the library and information science domain. Through meta-paths in this super-network, the influential strength of a paper and the awakening effect from neighboring papers can be quantified into 36 numerical features. By testing the effectiveness of these features in citation counts prediction, we try to prove our hypothesis. Thus an effective predictor in machine learning is trained upon these features. Using this predictor, we showed that most neighboring papers in the super-network had effects on future citation counts. The effectiveness of these features is again demonstrated through experiments on papers with different publication years. We also did a case study on papers that were significantly affected by the awakening effect, and found that the model proposed in this paper can also be used to explain some common phenomena in knowledge diffusion. All results show that the awakening effect could be not only ubiquitous but also quantifiable.

Predicting the citation count and CiteScore of journals one year in advance

Prediction of the future performance of academic journals is a task that can benefit a variety of stakeholders including editorial staff, publishers, indexing services, researchers, university administrators and granting agencies. Using historical data on journal performance, this can be framed as a machine learning regression problem. In this work, we study two such regression tasks: 1) prediction of the number of citations a journal will receive during the next calendar year, and 2) prediction of the Elsevier CiteScore a journal will be assigned for the next calendar year. To address these tasks, we first create a dataset of historical bibliometric data for journals indexed in Scopus. We propose the use of neural network models trained on our dataset to predict the future performance of journals. To this end, we perform feature selection and model configuration for a Multi-Layer Perceptron and a Long Short-Term Memory. Through experimental comparisons to heuristic prediction baselines and classical machine learning models, we demonstrate superior performance in our proposed models for the prediction of future citation and CiteScore values.