Which Keyword Extraction Method Performs Better for Emerging Technology Detection?

Abstract

Listen

Translate article

Detecting emerging technologies is a major concern for many researchers. In the process of emergence detection, we need to ha have some terms to assess them to be emergent or not. The demand of a method that can provide useful and meaningful terms but easy to implement made us to evaluate different automatic keyword extraction methods in the emergence detection process. In this study, we applied 11 automatic keyword extraction methods to detect emerging technologies in the field of Artificial Intelligence (AI) for the years between 2010 to 2019. These methods including Single Rank, Text Rank, Topical Page Rank, Position Rank, TFIDF, BERT, KPminer, RAKE, Topic Rank, RaKUn, and YAKE. All methods worked properly but RaKUn did not work well. Four methods of Single Rank, Text Rank, Topical Page Rank, and Position Rank performed better compared to other methods to capture more useful terms to be emergent. However, the BERT method was also successful in the process of emergence detection, especially for capturing highly emergent terms that other methods were not able to capture them. Therefore, using different methods such as Single Rank and BERT can be complementary. Additionally, the performance of the RAKE method to capture useful terms in a very fast way was also interesting, as it can be used in special situations where the data is big and the speed is important.