Unsupervised Keyphrase Extraction Research Articles

Extracting Key-phrase Embedding using Deep Average Network and Maximal Marginal Relevance to Enhance Information Retrieval

Background: Automatic keyphrase extraction (AKE) is essential to many NLP and information retrieval tasks. Extracting high-quality key phrases is difficult due to technological advancements and the exponential growth ‎of ‎textual data and digital sources. Unsupervised keyphrase extraction with cheap computing cost that relies ‎on ‎heuristic notions of phrase importance such as embedding similarities but their development necessitates in-depth subject expertise. Materials and Methods: This paper presents a method to obtain a semantic understanding of the query ‎and index documents by using ‎the embedding technique(universal Sentence ‎encoder (USE) ) while keeping the most informative ‎using Maximal Marginal Relevance (MMR) and then scoring(an ‎inverted index) the most documents relevant to the query ‎vector to improve the ‎performance of IR systems.‎ Results: The proposed retrieval model implement on the (Fire2011) dataset. The final ‎stage was evaluating the results of the baseline and the results (indexing and ranking) by using mean average precision (MAP). The ‎result of the baseline was 0.61, while the result ‎inverted index was 0.6277519 .‎ Conclusions: In this paper, we have discussed document retrieval using keep key phrases that ‎have ‏informativeness ‎properties by ‎using maximal ‎marginal ‎relevance, since if we extract a fixed number of top ‎keyphrases, ‎redundancy ‎hinders the diversification of the ‎extracted keyphrases.

Enhancing unsupervised keyphrase extraction through the integration of structural details in embedding-based approaches

Enhancing unsupervised keyphrase extraction through the integration of structural details in embedding-based approaches

AdaptiveUKE: Towards adaptive unsupervised keyphrase extraction with gated topic modeling

Keyphrase extraction aims to identify the purport-specific words or phrases from a free-format natural language document, which also requires the keyphrases to be concise and informative. Existing works mainly encode the document and phrases into the same vector space and score keyphrases directly employing similarity measures. However, such methods have two major limitations: (1) The semantic richness of documents in real-world scenarios can be diverse and it is tricky to model such diversity when representing a document with one specific embedding. (2) Using similarity measures alone to score keyphrases fails to detect phrases from all related topics, leading to semantics over-concentration of extracted results. To address these challenges, we propose AdaptiveUKE,22Our code is released at https://github.com/NLPCodebase/AdaptiveUKE. a simple yet effective model for adaptive unsupervised keyphrase extraction. Firstly, to adapt to the varying degree of document semantic richness, we propose a novel gated topic modeling strategy, allowing each topic to be assigned independently rather than in a competitive fashion. Secondly, we design a topic-guided scoring algorithm to extract keyphrases under unsupervised conditions. Fusing the importance and relatedness of each topic enables efficient ranking of candidate phrases. Finally, we conduct extensive experiments on six widely-used datasets. The results show that AdaptiveUKE achieves highly competitive results across several keyphrase extraction benchmarks, surpassing state-of-the-art on Inspec, DUC2001, SemEval2010 and reaching the current best performance on SemEval2017, Theses100, Krapivin2009.

MICRank: Multi-information interconstrained keyphrase extraction

Keyphrase Extraction is an automatic task that involves identifying the key words or phrases that capture the main content of an article. It is useful for various downstream tasks, including text search, text clustering, and text classification. Embedding-based methods for keyphrase extraction have shown promising results by utilizing pre-trained language models to represent candidate phrases and documents separately. These methods then rank the candidate phrases based on the cosine similarity between the document and the candidate phrases embeddings. However, there are mainly two shortcomings in such methods: I) Redundancy errors, when there are partial repetitions of candidate keyphrases, the methods tend to use redundant long phrases as keyphrases; II) Low keyphrase coverage, such as some keyphrases used to describe locally important information are ignored. In this paper, we propose an unsupervised keyphrase extraction method called “MICRank”, which evaluates the importance of candidate keyphrases from three perspectives: global information, local information, and attribute information, and solved the aforementioned issues. The experimental results on six benchmarks demonstrate that the proposed MICRank method outperforms the state-of-the-art unsupervised keyphrase extraction methods. In addition, this paper improves the judgment criterion of correct keyphrase extraction and introduces a new evaluation metric called S1@M (M ∈ {5,10,15}) to address the issue of synonyms being considered incorrect predictions.

Y-Rank: A Multi-Feature-Based Keyphrase Extraction Method for Short Text

Keyphrase extraction is a critical task in text information retrieval, which traditionally employs both supervised and unsupervised approaches. Supervised methods generally rely on large corpora, which introduce the problems of availability, while unsupervised methods are independent of out-sources but also lead to defects like imperfect statistical features or low accuracy. Particularly in short-text scenarios, limited text features often result in low-quality candidate ranking. To address this issue, this paper proposes Y-Rank, a lightweight unsupervised keyphrase extraction method that extracts the average information content of candidate sentences as the key statistical features from a single document, and follows a graph construction approach based on similarity to obtain the semantic features of keyphrase with high-quality and ranking accuracy. Finally, the top-ranked keyphrases are acquired by the fusion of these features. The experimental results on five datasets illustrate that Y-Rank outperforms the other nine unsupervised methods, achieves enhancements on six accuracy metrics, including Precision, Recall, F-Measure, MRR, MAP, and Bpref, and performs the highest improvement in short text scenarios.

Hybrid Approach To Unsupervised Keyphrase Extraction

The exponential growth of textual data poses a monumental challenge for extracting meaningful knowledge. Manually identifying descriptive keywords or keyphrases for each document is infeasible given the massive daily generated text. Automatic keyphrase extraction is, therefore, essential. However, current techniques struggle with learning the most salient semantic features from lengthy documents. This hybrid keyphrase extraction framework uniquely combines the complementary strengths of graph-based and textual feature methods. Our approach demonstrates improved performance over relying solely on statistical or graphical. Graph-based systems leverage word co- occurrence networks to score importance. Textual methods extract keyphrases using linguistic properties. Together, these complementary techniques overcome the limitations of relying on any strategy. The hybrid approach is evaluated on standard SemEval 2017 Task 10 and SemEval 2010 Task 5 benchmark datasets for scientific paper keyphrase extraction. Performance is quantified using the F1 score relative to human-annotated ground truth keyphrase. Results will quantify effectiveness on long documents with thousands of terms where only a few keywords represent salient concepts. Results show our technique effectively identifies the most salient semantic keywords, overcoming limitations of current techniques that struggle to mix features of graphical or statistical methods. Our experiments demonstrate that the proposed hybrid approach achieves superior F1 scores compared to current state-of-the-art methods on benchmark datasets. These results validate that synergistically combining graph and textual features enables more accurate keyphrase extraction, especially for long documents laden with extraneous terms.

Contextual topic discovery using unsupervised keyphrase extraction and hierarchical semantic graph model

Recent technological advancements have led to a significant increase in digital documents. A document’s key information is generally represented by the keyphrases that provide the abstract description contained therein. With traditional keyphrase techniques, however, it is difficult to identify relevant information based on context. Several studies in the literature have explored graph-based unsupervised keyphrase extraction techniques for automatic keyphrase extraction. However, there is only limited existing work that embeds contextual information for keyphrase extraction. To understand keyphrases, it is essential to grasp both the concept and the context of the document. Hence, a hybrid unsupervised keyphrase extraction technique is presented in this paper called ContextualRank, which embeds contextual information such as sentences and paragraphs that are relevant to keyphrases in the keyphrase extraction process. We propose a hierarchical topic modeling approach for topic discovery based on aggregating the extracted keyphrases from ContextualRank. Based on the evaluation on two short-text datasets and one long-text dataset, ContextualRank obtains remarkable improvements in performance over other baselines in the short-text datasets.

Towards unsupervised keyphrase extraction via an autoregressive approach

Keyphrase extraction is a technique used to capture the core information of documents and is an upstream task for advanced information retrieval systems, particularly in the academic realm. Current unsupervised methods are primarily built on a score-and-rank framework with a consistent inability to acquire mutual information between extracted keyphrases, especially with graph-based models. Utilizing the autoregressive structure that is typically used in sequence-to-sequence text generation models, we propose a plug-and-play optimizer named C-Decay that can be integrated into any graph-based unsupervised keyphrase extraction model for a stable performance boost, and that mitigates the bias of certain semantically or lexically dominant tokens by optimizing the origin score distribution output by graph-based models directly. The architecture of C-Decay includes the keyphrase pool, the gain vector and the decay factor, where the keyphrase pool is designed to realize an autoregressive structure and the gain vector and the decay factor are the optimization operator. Herein, we examine three graph-based models integrated with C-Decay, and the experiment is conducted on four datasets KDD, Semeval, Nguyen, and Krapivin. Moreover, we prove that C-Decay can improve accuracy and F-Measure by an average of approximately 50% and 20%, respectively.

Improving unsupervised keyphrase extraction by modeling hierarchical multi-granularity features

Existing unsupervised keyphrase extraction methods typically emphasize the importance of the candidate keyphrase itself, ignoring other important factors such as the influence of uninformative sentences. We hypothesize that the salient sentences of a document are particularly important as they are most likely to contain keyphrases, especially for long documents. To our knowledge, our work is the first attempt to exploit sentence salience for unsupervised keyphrase extraction by modeling hierarchical multi-granularity features. Specifically, we propose a novel position-aware graph-based unsupervised keyphrase extraction model, which includes two model variants. The pipeline model first extracts salient sentences from the document, followed by keyphrase extraction from the extracted salient sentences. In contrast to the pipeline model which models multi-granularity features in a two-stage paradigm, the joint model accounts for both sentence and phrase representations of the source document simultaneously via hierarchical graphs. Concretely, the sentence nodes are introduced as an inductive bias, injecting sentence-level information for determining the importance of candidate keyphrases. We compare our model against strong baselines on three benchmark datasets including Inspec, DUC 2001, and SemEval 2010. Experimental results show that the simple pipeline-based approach achieves promising results, indicating that keyphrase extraction task benefits from the salient sentence extraction task. The joint model, which mitigates the potential accumulated error of the pipeline model, gives the best performance and achieves new state-of-the-art results while generalizing better on data from different domains and with different lengths. In particular, for the SemEval 2010 dataset consisting of long documents, our joint model outperforms the strongest baseline UKERank by 3.48%, 3.69% and 4.84% in terms of F1@5, F1@10 and F1@15, respectively. We also conduct qualitative experiments to validate the effectiveness of our model components.

Generative non-autoregressive unsupervised keyphrase extraction with neural topic modeling

Generative non-autoregressive unsupervised keyphrase extraction with neural topic modeling

Keyphrase Extraction by Improving TextRank with an Integration of Word Embedding and Syntactic Information

Background: As a known key phrase extraction algorithm, TextRank is an analogue of PageRank algorithm, which relied heavily on the statistics of term frequency in the manner of co-occurrence analysis. Objective: The frequency-based characteristic made it a neck-bottle for performance enhancement, and various improved TextRank algorithms were proposed in the recent years. Most of improvements incorporated semantic information into key phrase extraction algorithm and achieved improvement. Method: In this research, taking both syntactic and semantic information into consideration, we integrated syntactic tree algorithm and word embedding and put forward an algorithm of Word Embedding and Syntactic Information Algorithm (WESIA), which improved the accuracy of the TextRank algorithm. Results: By applying our method on a self-made test set and a public test set, the result implied that the proposed unsupervised key phrase extraction algorithm outperformed the other algorithms to some extent.

Attention-based Unsupervised Keyphrase Extraction and Phrase Graph for COVID-19 Medical Literature Retrieval

Searching, reading, and finding information from the massive medical text collections are challenging. A typical biomedical search engine is not feasible to navigate each article to find critical information or keyphrases. Moreover, few tools provide a visualization of the relevant phrases to the query. However, there is a need to extract the keyphrases from each document for indexing and efficient search. The transformer-based neural networks—BERT has been used for various natural language processing tasks. The built-in self-attention mechanism can capture the associations between words and phrases in a sentence. This research investigates whether the self-attentions can be utilized to extract keyphrases from a document in an unsupervised manner and identify relevancy between phrases to construct a query relevancy phrase graph to visualize the search corpus phrases on their relevancy and importance. The comparison with six baseline methods shows that the self-attention-based unsupervised keyphrase extraction works well on a medical literature dataset. This unsupervised keyphrase extraction model can also be applied to other text data. The query relevancy graph model is applied to the COVID-19 literature dataset and to demonstrate that the attention-based phrase graph can successfully identify the medical phrases relevant to the query terms.

Keyphrase Extraction on Covid-19 Tweets Based on Doc2Vec and YAKE

Keyword and keyphrase extraction are one of the initial foundations for performing several text processing operations such as summarization and document clustering. YAKE is one of the techniques used for unsupervised and independent keyphrase extraction, it does not require a corpus for linguistic tools such as NER and POS-tag. However, the use of YAKE in microblogging documents such as Twitter often results in a keyphrase that is less representative because of the lack of words used for ranking. This paper offers a solution to this problem by looking for similar tweets in the keyphrase extraction process using Doc2Vec so that the number of words used in the YAKE ranking process can be greater. Covid-19 tweets related are used as dataset as the topic is currently widely discussed on social media to prove that the proposed approach could improve keyphrase extraction performance

ISKE: An unsupervised automatic keyphrase extraction approach using the iterated sentences based on graph method

The automatic extraction of key information is necessary for knowledge discovery in this era of rapid knowledge growth. The extraction of key information can also help researchers quickly obtain the information they want instead of reading through all potential documents. Recently, researchers have refocused their attention from words to sentences because utilizing sentences outperforms with respect to illustrating semantics and reduces the calculation complexity. We present a novel and lightweight automatic keyphrase extraction algorithm that does not depend on any external resources, including an external dictionary or corpus. Unlike traditional graph-based algorithms that iterate words to generate keyphrase lists, our proposal uses iterated sentences to rank words and generate keyphrase lists for the semantic information of sentences that are more complete than the word. We initialize the values of words with weighted information and generate a sentence score using these values. Then, we integrate sentences to update their values; hence, the values of the words are updated with the sentence information. We iterate this process until the values of the sentences and words converge. The proposed method is based on a measurement of the relations between sentences and an evaluation of the flow of these relations in an easily understood manner. These relationships are based on the hypothesis that the causality between adjacent sentences is semantically stronger than the causality between words. We not only increase the extraction accuracy, but also reduce the number of iterations of the algorithm. We compare our proposed method with five strong, popular baseline algorithms on four datasets. The results show that our proposed method performs better than the other algorithms on three evaluation metrics.

Automatic Keyphrase Extraction from Scientific Chinese Medical Abstracts Based on Character-Level Sequence Labeling

AbstractPurposeAutomatic keyphrase extraction (AKE) is an important task for grasping the main points of the text. In this paper, we aim to combine the benefits of sequence labeling formulation and pretrained language model to propose an automatic keyphrase extraction model for Chinese scientific research.Design/methodology/approachWe regard AKE from Chinese text as a character-level sequence labeling task to avoid segmentation errors of Chinese tokenizer and initialize our model with pretrained language model BERT, which was released by Google in 2018. We collect data from Chinese Science Citation Database and construct a large-scale dataset from medical domain, which contains 100,000 abstracts as training set, 6,000 abstracts as development set and 3,094 abstracts as test set. We use unsupervised keyphrase extraction methods including term frequency (TF), TF-IDF, TextRank and supervised machine learning methods including Conditional Random Field (CRF), Bidirectional Long Short Term Memory Network (BiLSTM), and BiLSTM-CRF as baselines. Experiments are designed to compare word-level and character-level sequence labeling approaches on supervised machine learning models and BERT-based models.FindingsCompared with character-level BiLSTM-CRF, the best baseline model with F1 score of 50.16%, our character-level sequence labeling model based on BERT obtains F1 score of 59.80%, getting 9.64% absolute improvement.Research limitationsWe just consider automatic keyphrase extraction task rather than keyphrase generation task, so only keyphrases that are occurred in the given text can be extracted. In addition, our proposed dataset is not suitable for dealing with nested keyphrases.Practical implicationsWe make our character-level IOB format dataset of Chinese Automatic Keyphrase Extraction from scientific Chinese medical abstracts (CAKE) publicly available for the benefits of research community, which is available at:https://github.com/possible1402/Dataset-For-Chinese-Medical-Keyphrase-Extraction.Originality/valueBy designing comparative experiments, our study demonstrates that character-level formulation is more suitable for Chinese automatic keyphrase extraction task under the general trend of pretrained language models. And our proposed dataset provides a unified method for model evaluation and can promote the development of Chinese automatic keyphrase extraction to some extent.

TripleRank: An unsupervised keyphrase extraction algorithm

Automatic keyphrase extraction algorithms aim to identify words and phrases that contain the core information in documents. As online scholarly resources have become widespread in recent years, better keyphrase extraction techniques are required to improve search efficiency. We present two features, keyphrase semantic diversity and keyphrase coverage, to overcome limitations of existing methods for unsupervised keyphrase extraction. Keyphrase semantic diversity is the degree of semantic variety in the extraction result, which is introduced to avoid extracting synonym phrases that contain the same high-score candidate. Keyphrase coverage refers to candidates’ representativeness of other words in documents. We propose an unsupervised keyphrase extraction method called TripleRank, which evaluates three features: word position (a sensitive feature for academic documents) and two innovative features mentioned above. The architecture of TripleRank includes three sub-models that score the three features and a summing model. Though involving multiple models, there is no typical iteration process in TripleRank; hence, the computational cost is relatively low. TripleRank has led the experiment results on four academic datasets compared to four state-of-the-art baseline models, which confirmed the influence of keyphrase semantic diversity and keyphrase coverage and proved the efficiency of our method.

Recommending Research Articles: A Multi-Level Chronological Learning-Based Approach Using Unsupervised Keyphrase Extraction and Lexical Similarity Calculation

Recommending Research Articles: A Multi-Level Chronological Learning-Based Approach Using Unsupervised Keyphrase Extraction and Lexical Similarity Calculation

A Review of Unsupervised Keyphrase Extraction Methods Using Within-Collection Resources

An essential part of a text generation task is to extract critical information from the text. People usually obtain critical information in the text via manual extraction; however, the asymmetry between the ability to process information manually and the speed of information growth makes it impossible. This problem can be solved by automatic keyphrase extraction. In this paper, the mainstream unsupervised methods to extract keyphrases are summarized, and we analyze in detail the reasons for the differences in the performance of methods then provided some solutions.

Semantic Unsupervised Automatic Keyphrases Extraction by Integrating Word Embedding with Clustering Methods

Increasingly, the web produces massive volumes of texts, alone or associated with images, videos, photographs, together with some metadata, indispensable for their finding and retrieval. Keywords/keyphrases that characterize the semantic content of documents should be, automatically or manually, extracted, and/or associated with them. The paper presents a novel method to address the problem of the automatic unsupervised extraction of keywords/phrases from texts, expressed both in English and in Italian. The main feature of this approach is the integration of two methods that have given interesting results: word embedding models, such as Word2Vec or GloVe able to capture the semantics of words and their context, and clustering algorithms, able to identify the essence of the terms and choose the more significant one(s), to represent the contents of a text. In the paper, the datasets used are presented, together with the method implemented and the results obtained. These results will be discussed, commented, and compared with those obtained in previous experimentations, using TextRank, Rapid Automatic Keyword Extraction (RAKE), and TF-IDF.

TeKET: a Tree-Based Unsupervised Keyphrase Extraction Technique

Automatic keyphrase extraction techniques aim to extract quality keyphrases for higher level summarization of a document. Majority of the existing techniques are mainly domain-specific, which require application domain knowledge and employ higher order statistical methods, and computationally expensive and require large train data, which is rare for many applications. Overcoming these issues, this paper proposes a new unsupervised keyphrase extraction technique. The proposed unsupervised keyphrase extraction technique, named TeKET or Tree-based Keyphrase Extraction Technique, is a domain-independent technique that employs limited statistical knowledge and requires no train data. This technique also introduces a new variant of a binary tree, called KeyPhrase Extraction (KePhEx) tree, to extract final keyphrases from candidate keyphrases. In addition, a measure, called Cohesiveness Index or CI, is derived which denotes a given node’s degree of cohesiveness with respect to the root. The CI is used in flexibly extracting final keyphrases from the KePhEx tree and is co-utilized in the ranking process. The effectiveness of the proposed technique and its domain and language independence are experimentally evaluated using available benchmark corpora, namely SemEval-2010 (a scientific articles dataset), Theses100 (a thesis dataset), and a German Research Article dataset, respectively. The acquired results are compared with other relevant unsupervised techniques belonging to both statistical and graph-based techniques. The obtained results demonstrate the improved performance of the proposed technique over other compared techniques in terms of precision, recall, and F1 scores.