Carbon-dating Articles with Transformer Series

Abstract

In the last decade, we witnessed significant technological advancements that had huge impacts across all different aspect of everyday life. Innovative technologies present important topics that need to be overseen from their early beginnings to be able to capture their breakthrough and prepare organizations for needed transitions. This research tries to detect weak signals of emerging technologies based on scientific articles published over the last decade covering several categories.To capture emerging technology trends, topics, we propose a method we refer to as "Carbon-dating Articles with Transformer Series". The name references the Radiocarbon dating (also referred to as carbon dating) method for determining the age of an object containing organic material by using the properties of radiocarbon, a radioactive isotope of carbon. In our method, we take advantage of the state-of-the-art transformer-based architectures in Natural Language Processing (NLP). A time-series of NLP models is trained to recognize new topics (adding fresh "radio-active" isotopes), while slowly forgetting past topics ("isotope half-life"). Using the trained time-series NLP models, scientific article signatures (i.e., text sequences) can be measured and positioned in time.We believe our "Carbon-dating Articles with Transformer Series" method can give valuable insights about when topics emerge and are picked-up by the larger community, timing of (published) articles presenting progressive research versus repeating research, and the bigger picture when combined with other insights like affiliations with nations, organizations, or academics.