Citation Recommendation Research Articles

ICA-CRMAS: Intelligent Context-Awareness Approach for Citation Recommendation based on Multi-Agent System

Navigating the ever-expanding sea of scientific literature presents a daunting challenge for researchers seeking relevant and up-to-date information. Traditional citation recommendation systems, while well-intentioned, often fall short due to their limited focus on text-based features and lack of contextual awareness. In this paper we introduce the ICA-CRMAS (Intelligent Context-Aware Approach for Citation Recommendation based on Multi-Agent System), an intelligent system that leverages the power of deep learning, semantic analysis, and multimodal learning to overcome these limitations. ICA-CRMAS goes beyond the surface, delving into the rich tapestry of information within academic papers, including figures, which often hold vital contextual clues. By weaving this contextual data directly into its recommendation models, ICA-CRMAS generates highly personalized and relevant suggestions. This comprehensive approach unlocks enhanced accuracy, diversity, and serendipity, enabling researchers to effectively discover papers aligning with their interests and research objectives. ICA-CRMAS illuminates its reasoning. Instead of opaque suggestions, the system provides clear explanations that justify and illustrate recommended citations. This transparency builds user confidence, allowing researchers to critically engage with and trust the system’s recommendations. Evaluation experiments conducted on real-world academic datasets demonstrate that ICA-CRMAS outperforms existing approaches across various metrics. it surpassing its closest competitor by a margin of 7.53 on accuracy, 6.07% on MRR and by 5.87 on Recall. User feedback further reinforces its effectiveness, with an Overall System Usability Scale (SUS) score of 76.73, exceeding benchmark scores for comparable systems.

Personalized global citation recommendation with diversification awareness

Personalized global citation recommendation with diversification awareness

ICRM: An intelligent citation recommendation mechanism based on BERT and weighted BoW models

With the field of technology has witnessed rapid advancements, attracting an ever-growing community of researchers dedicated to developing theories and techniques. This paper proposes an innovative ICRM (Intelligent Citation Recommendation Mechanism), designed to automate the process of suggesting the appropriate number of citations for individual brackets within a document. The proposed ICRM comprises three phases: Coarse-grained Weighted Bag of Word (WCBW), Fine-grained SciBERT (FSB) and Citation Adjustment phases. Firstly, the WCBW phase employs TF-IDF to extract keywords from both target and candidate documents, forming vectors that capture word significance along with metadata like authorship, keywords, and titles. It aims to identify relevant papers from a database, serving as initial candidates for each bracket. Secondly, the FSB phase employs the SciBERT model to assess the similarity between candidate documents and the local context around brackets, enhancing the precision of recommendations. It refines this selection by analyzing candidate-document relationships within the proximity of the brackets. Lastly, the Citation Adjustment phase tackles overlapping citations and ensures that recommended citation numbers align with user-defined criteria, resolving issues of imbalance. The simulation results demonstrate that the proposed ICRM outperforms existing models significantly in terms of precision, recall and F1-score.

Evolving Knowledge Graph Representation Learning with Multiple Attention Strategies for Citation Recommendation System

The growing number of publications in the field of artificial intelligence highlights the need for researchers to enhance their efficiency in searching for relevant articles. Most paper recommendation models either rely on simplistic citation relationships among papers or focus on content-based approaches, both of which overlook interactions within academic networks. To address the aforementioned problem, knowledge graph embedding (KGE) methods have been used for citation recommendations because recent research proves that graph representations can effectively improve recommendation model accuracy. However, academic networks are dynamic, leading to changes in the representations of users and items over time. The majority of KGE-based citation recommendations are primarily designed for static graphs, thus failing to capture the evolution of dynamic knowledge graph (DKG) structures. To address these challenges, we introduced the evolving knowledge graph embedding (EKGE) method. In this methodology, evolving knowledge graphs are input into time-series models to learn the patterns of structural evolution. The model has the capability to generate embeddings for each entity at various time points, thereby overcoming limitation of static models that require retraining to acquire embeddings at each specific time point. To enhance the efficiency of feature extraction, we employed a multiple attention strategy. This helped the model find recommendation lists that are closely related to a user’s needs, leading to improved recommendation accuracy. Various experiments conducted on a citation recommendation dataset revealed that the EKGE model exhibits a 1.13% increase in prediction accuracy compared to other KGE methods. Moreover, the model’s accuracy can be further increased by an additional 0.84% through the incorporation of an attention mechanism.

Enhancing local citation recommendation with recurrent highway networks and SciBERT-based embedding

When writing academic papers, referencing statements, claims and previous studies is always an important activity. However, it is considered challenging for scientists to find relevant and appropriate scientific articles which are closely related to their current works in order to reference them in their research. As a consequence of the rapid growth in scientific papers being published every year, researchers might easily get overwhelmed within a huge number of resources. One way to help them find the desired references more easily is to use context-aware citation recommendation. The citation recommender system can automatically provide a list of suitable papers as references based on specified inputs which reflect the researchers’ interests. Among the outstanding achievements of deep learning and natural language processing in recent years, the utilization of deep neural learning architectures have supported to address the problem of citation recommendation. As the result, the neural citation recommendation area has received much attention from the academic community, with the aim of enhancing the precision and correctness of the results of existing citation recommendation systems. Following this research direction, in our paper we present a novel context-aware citation recommendation model, called RHN-DualLCR (Recurrent Highway Networks – Dual Local Citation Recommendation), which integrates Recurrent Highway Networks (RHN), an improved model of the original Bidirectional Long Short-Term Memory (BiLSTM) architecture, and uses a SciBERT-based (Science text of Bidirectional Encoder Representations from Transformers) embedding layer to build up the efficiency of the state-of-the-art local citation recommendation model, which enriches context representation with global information. Our research demonstrates its originality and relevance because we used one of the latest achievements of deep models (RHN model) and natural language processing (SciBERT) applied to the citation recommendation problem. We have conducted experiments on the RHN-DualLCR model on 3 widely known datasets for the citation recommendation problem: ACL-200 (Association for Computational Linguistics), ACL-600 and RefSeer, and also used 2 common evaluation standards Mean Reciprocal Rank (MRR) and the Recall@K (R@K for short) to evaluate the performance of our model. Experimental results show that our proposed model is 3% to 16% better than the original models or state-of-the-art models.

A fast local citation recommendation algorithm scalable to multi-topics

In the era of rapid paper publications in various venues, automatic citation recommendations would be highly useful to researchers when they write papers. Local citation recommendation aims to recommend possible papers to cite given local citation contexts. Previous work mainly computes the similarity score between citation contexts and cited papers on a one-to-one basis, which is quite time-consuming. We train a pair of neural network encoders that map citation contexts and all possible cited papers to the same vector space, respectively. After that, we index the positions of all cited papers in the vector space. This makes our process for searching recommended papers considerably faster. On the other hand, existing methods tend to recommend papers that are highly similar to each other, which makes recommendations lack diversity. Therefore, we extend our algorithm to perform multi-topic recommendations. We generate multi-topic training examples based on the index we mentioned earlier. Furthermore, we specially design a multi-group contrastive learning method to train our model so that it can distinguish different topics. Empirical experiments show that our model outperforms previous methods by a wide margin. Our model is also light weighted and has been deployed online so that researchers can use it to obtain recommended citations for their own paper in real-time.

Improving Wikipedia verifiability with AI

Verifiability is a core content policy of Wikipedia: claims need to be backed by citations. Maintaining and improving the quality of Wikipedia references is an important challenge and there is a pressing need for better tools to assist humans in this effort. We show that the process of improving references can be tackled with the help of artificial intelligence (AI) powered by an information retrieval system and a language model. This neural-network-based system, which we call SIDE, can identify Wikipedia citations that are unlikely to support their claims, and subsequently recommend better ones from the web. We train this model on existing Wikipedia references, therefore learning from the contributions and combined wisdom of thousands of Wikipedia editors. Using crowdsourcing, we observe that for the top 10% most likely citations to be tagged as unverifiable by our system, humans prefer our system’s suggested alternatives compared with the originally cited reference 70% of the time. To validate the applicability of our system, we built a demo to engage with the English-speaking Wikipedia community and find that SIDE’s first citation recommendation is preferred twice as often as the existing Wikipedia citation for the same top 10% most likely unverifiable claims according to SIDE. Our results indicate that an AI-based system could be used, in tandem with humans, to improve the verifiability of Wikipedia.

Multi-information fusion based on dual attention and text embedding network for local citation recommendation

Multi-information fusion based on dual attention and text embedding network for local citation recommendation

Deep Learning-Assisted Citation Recommendation System Using Multi- cell RNN Approach

A process of retrieving relevant or suitable papers that are mostly related to the research topic while carrying research knowing the behavior and patterns of citation for mapping the disciplines and subjects. This is used increasingly for judging usually takes a huge amount of effort and time. The citation recommendation is used frequently for solving this issue on suggesting the list of candidate papers automatically which may match the references of user. On applying Deep learning (DL) models in the citation recommendation system, the performance enhancement is greatly visible in various recent research studies. Hence, in this work, a deep learning-based model is employed for the citation recommendation system. Primarily, the input dataset is taken and the function words are removed using stochastic matching pattern scheme at which the library is created. Then, the canonicalization of document is carried by means of probabilistic min-match algorithm. The data is classified finally using Multi-cell RNN (Recurrent neural network) approach after which the rank is set for citation. At last, the performance is assessed for various metrics like MAP, MRR, precision, recall, and F1-measure. The obtained results are then compared with traditional models to validate and show the enhancement of proposed model over existing schemes. It is revealed from the analysis that the proposed scheme shows improvement in results.

Use of artificial intelligence in biblical citation recommendations in the New Testament

Religion occupies a prominent place in people’s daily lives and is made explicit to the public or the faithful through preaching or exposition of their sacred texts. The Holy Bible is the religious literature of Christianity, and its text has a unique nature of interpretation and knowledge extraction, that is, through the reading done by specialists (theologians). However, an automated knowledge extraction or that involves some automatic mechanism intelligence to support the interpretation (hermeneutics) of the Biblical text is not observed in the literature. Probably this gap in the literature is caused by the complexity of the biblical textual corpus and the multiplicity of genres it has, being an interpretative challenge even for human specialists. Therefore, this article primarily seeks to build an automated way through artificial intelligence (AI) to provide contextual biblical quotations from the four gospels of the New Testament for the construction of sermons or development of homiletics, which is the art of producing religious sermons for teaching and interpretation of the Biblical message. The methodology used in this article seeks to employ artificial intelligence techniques to implement the proposed solution, that is, a hybrid recommendation system to quote texts from Biblical passages. The AI techniques involved are text mining, natural language processing and supervised learning. Secondarily, this work aims to verify whether the combination of natural language processing techniques and machine learning can provide subsidies for the recovery or extraction of knowledge from complex textual corpus analogous to the biblical corpus. The results show that the proposed hybrid recommendation system is capable of extracting semantic and contextual meaning from the Biblical text, fundamental in the construction of homiletics. The performance evaluation metrics indicate the robustness of the results and consequently validate the findings of this research. Therefore, the combination of these techniques can be extrapolated by the scientific community to aid in the interpretive recovery of complex textual corpus.

Context-aware citation recommendation of scientific papers: comparative study, gaps and trends

Context-aware citation recommendation of scientific papers: comparative study, gaps and trends

A Systematic Review of Citation Recommendation Over the Past Two Decades

A citation is a reference to the source of information used in an article. Citations are very useful for students and researchers to locate relevant information on a topic. Proper citation is also important in the academic ethics of article writing. Due to the rapid growth of scientific works published each year, how to automatically recommend citations to students and researchers has become an interesting but challenging research problem. In particular, a citation recommendation system can assist students to identify relevant papers and literature for academic writing. Citation recommendation can be classified into local and global citation recommendation depending on whether a specific local citation context is given; e.g., the text surrounding a citation placeholder. This article provides a systematic review on global citation recommendation models and compares the reviewed methods from the traditional topic- based models to the recent models embedded with deep neural networks, aiming to summarize this field to facilitate researchers working on citation recommendation.

Enhanced context-aware citation recommendation with auxiliary textual information based on an auto-encoding mechanism

Enhanced context-aware citation recommendation with auxiliary textual information based on an auto-encoding mechanism

Research on semantic representation and citation recommendation of scientific papers with multiple semantics fusion

Research on semantic representation and citation recommendation of scientific papers with multiple semantics fusion

A Scientific Paper Recommendation Framework Based on Multi-Topic Communities and Modified PageRank

Personalized PageRank is a variant of PageRank, widely developed for citation recommendation. However, the personalized PageRank that works with a vast amount and rich scholarly data still results in information overload. Sometimes, junior scholars still need help to arrange queries quickly because of limited domain knowledge. Senior researchers need reference papers regarding a similar topic they intend to search for and related topics as a new insight. In this research, scientific citation recommendation aims to find the most influential papers with similar and related topics. Related topic papers in serendipitous perspectives are reference papers that are novel, diversified and unexpected to a user. The unexpectedness of recommended papers can be papers with different topics to queries but still relevant. To accomplish these challenges, we propose a framework of scientific citation recommendation with serendipitous perspectives. The framework includes feature extraction of an academic citation network, selection of multi-topic communities, and ranking papers in the selected multi-topic communities by modified PageRank. Papers in the chosen communities tend to link to similar and related papers. Modified PageRank is an extension of personalized PageRank, which works on multi-topic communities and manuscript queries. The experiments reveal that the proposed models outperform some models of personalized PageRank and some models of Content-Based Filtering. The multi-topic communities-based models work more effectively than the baselines if they run in a large dataset since the topic communities become more cohesive.

MP-BERT4REC: Recommending Multiple Positive Citations for Academic Manuscripts via Content-Dependent BERT and Multi-Positive Triplet

Considering the rapidly increasing number of academic papers, searching for and citing appropriate references has become a non-trial task during the wiring of papers. Recommending a handful of candidate papers to a manuscript before publication could ease the burden of the authors, and help the reviewers to check the completeness of the cited resources. Conventional approaches on citation recommendation generally consider recommending one ground-truth citation for a query context from an input manuscript, but lack of consideration on co-citation recommendations. However, a piece of context often needs to be supported by two or more co-citation pairs. Here, we propose a novel scientific paper modeling for citation recommendations, namely Multi-Positive BERT Model for Citation Recommendation (MP-BERT4CR), complied with a series of Multi-Positive Triplet objectives to recommend multiple positive citations for a query context. The proposed approach has the following advantages: First, the proposed multi-positive objectives are effective to recommend multiple positive candidates. Second, we adopt noise distributions which are built based on the historical co-citation frequencies, so that MP-BERT4CR is not only effective on recommending high-frequent co-citation pairs; but also the performances on retrieving the low-frequent ones are significantly improved. Third, we propose a dynamic context sampling strategy which captures the ``macro-scoped'' citing intents from a manuscript and empowers the citation embeddings to be content-dependent, which allow the algorithm to further improve the performances. Single and multiple positive recommendation experiments testified that MP-BERT4CR delivered significant improvements. In addition, MP-BERT4CR are also effective in retrieving the full list of co-citations, and historically low-frequent co-citation pairs compared with the prior works.

Heterogeneous deep graph convolutional network with citation relational BERT for COVID-19 inline citation recommendation

The outbreak of COVID-19 brings almost the biggest explosions of scientific literature ever. Facing such volume literature, it is hard for researches to find desired citation when carrying out COVID-19 related research, especially for junior researchers. This paper presents a novel neural network based method, called citation relational BERT with heterogeneous deep graph convolutional network (CRB-HDGCN), for COVID-19 inline citation recommendation task. The CRB-HDGCN contains two main stages. The first stage is to enhance the representation learning of BERT model for COVID-19 inline citation recommendation task through CRB. To achieve the above goal, an augmented citation sentence corpus, which replaces the citation placeholder with the title of the cited papers, is used to lightly retrain BERT model. In addition, we extract three types of sentence pair according citation relation, and establish sentence prediction tasks to further fine-tune the BERT model. The second stage is to learn effective dense vector of nodes among COVID-19 bibliographic graph through HDGCN. The HDGCN contains four layers which are essentially all sub neural networks. The first layer is initial embedding layer which generates initial input vectors with fixed size through CRB and a multilayer perceptron. The second layer is a heterogeneous graph convolutional layer. In this layer, we expand traditional homogeneous graph convolutional network into heterogeneous by subtly adding heterogeneous nodes and relations. The third layer is a deep attention layer. This layer uses trainable project vectors to reweight the node importance simultaneously according to both node types and convolution layers, which further promotes the performance of learnt node vectors. The last decoder layer recovers the graph structure and let the whole network trainable. The recommendation is finally achieved by integrating the high performance heterogeneous vectors learnt from CRB-HDGCN with the query vectors. We conduct experiments on the CORD-19 and LitCovid datasets. The results show that compared with the second best method CO-Search, CRB-HDGCN improves MAP, MRR, P@100 and R@100 with 21.8%, 22.7%, 37.6% and 21.2% on CORD-19, and 29.1%, 25.9%, 15.3% and 11.3% on LitCovid, respectively.

Legal case document similarity: You need both network and text

Estimating the similarity between two legal case documents is an important and challenging problem, having various downstream applications such as prior-case retrieval and citation recommendation. There are two broad approaches for the task — citation network-based and text-based. Prior citation network-based approaches consider citations only to prior-cases (also called precedents) (PCNet). This approach misses important signals inherent in Statutes (written laws of a jurisdiction). In this work, we propose Hier-SPCNet that augments PCNet with a heterogeneous network of Statutes. We incorporate domain knowledge for legal document similarity into Hier-SPCNet, thereby obtaining state-of-the-art results for network-based legal document similarity.Both textual and network similarity provide important signals for legal case similarity; but till now, only trivial attempts have been made to unify the two signals. In this work, we apply several methods for combining textual and network information for estimating legal case similarity. We perform extensive experiments over legal case documents from the Indian judiciary, where the gold standard similarity between document-pairs is judged by law experts from two reputed Law institutes in India. Our experiments establish that our proposed network-based methods significantly improve the correlation with domain experts’ opinion when compared to the existing methods for network-based legal document similarity. Our best-performing combination method (that combines network-based and text-based similarity) improves the correlation with domain experts’ opinion by 11.8% over the best text-based method and 20.6% over the best network-based method. We also establish that our best-performing method can be used to recommend/retrieve citable and similar cases for a source (query) case, which are well appreciated by legal experts.

DisenCite: Graph-Based Disentangled Representation Learning for Context-Specific Citation Generation

Citing and describing related literature are crucial to scientific writing. Many existing approaches show encouraging performance in citation recommendation, but are unable to accomplish the more challenging and onerous task of citation text generation. In this paper, we propose a novel disentangled representation based model DisenCite to automatically generate the citation text through integrating paper text and citation graph. A key novelty of our method compared with existing approaches is to generate context-specific citation text, empowering the generation of different types of citations for the same paper. In particular, we first build and make available a graph enhanced contextual citation dataset (GCite) with 25K edges in different types characterized by citation contained sections over 4.8K research papers. Based on this dataset, we encode each paper according to both textual contexts and structure information in the heterogeneous citation graph. The resulted paper representations are then disentangled by the mutual information regularization between this paper and its neighbors in graph. Extensive experiments demonstrate the superior performance of our method comparing to state-of-the-art approaches. We further conduct ablation and case studies to reassure that the improvement of our method comes from generating the context-specific citation through incorporating the citation graph.

Dual Attention Model for Citation Recommendation with Analyses on Explainability of Attention Mechanisms and Qualitative Experiments

Abstract Based on an exponentially increasing number of academic articles, discovering and citing comprehensive and appropriate resources have become non-trivial tasks. Conventional citation recommendation methods suffer from severe information losses. For example, they do not consider the section header of the paper that the author is writing and for which they need to find a citation, the relatedness between the words in the local context (the text span that describes a citation), or the importance of each word from the local context. These shortcomings make such methods insufficient for recommending adequate citations to academic manuscripts. In this study, we propose a novel embedding-based neural network called dual attention model for citation recommendation (DACR) to recommend citations during manuscript preparation. Our method adapts the embedding of three semantic pieces of information: words in the local context, structural contexts,1 and the section on which the author is working. A neural network model is designed to maximize the similarity between the embedding of the three inputs (local context words, section headers, and structural contexts) and the target citation appearing in the context. The core of the neural network model comprises self-attention and additive attention; the former aims to capture the relatedness between the contextual words and structural context, and the latter aims to learn their importance. Recommendation experiments on real-world datasets demonstrate the effectiveness of the proposed approach. To seek explainability on DACR, particularly the two attention mechanisms, the learned weights from them are investigated to determine how the attention mechanisms interpret “relatedness” and “importance” through the learned weights. In addition, qualitative analyses were conducted to testify that DACR could find necessary citations that were not noticed by the authors in the past due to the limitations of the keyword-based searching.