Discovering Associations Among Technologies Using Neural Networks for Tech-Mining

Abstract

In both public and private sectors, critical technology-based tasks, such as innovation, forecasting, and road-mapping, are faced with unmanageable complexity due to the ever-expanding web of technologies which can range into thousands. This context cannot be easily handled manually or with efficient speed. However, more precise and insightful answers are expected. These answers are the fundamental challenge addressed by tech-mining. For tech-mining, discovering the associations among them is a critical task. These associations are used to form a weighted directed graph of technologies called “association tech-graph” which is used for technology development, trend analysis, policymaking, strategic planning, and innovation. In this article, we present a novel method to build an artificial intelligence (AI) agent for automatic association discovery among technologies in a way that matches the quality of the human experts. To this end, neural network-based word embedding methods are exploited to represent technology terms as vectors, and their associations are calculated using similarity measures. To increase the accuracy of the vectors, several crawlers are built to acquire more appropriate training data. Furthermore, we introduce a validation method to measure the accuracy of the AI agent compared to human intelligence, which allows us to discuss the drawbacks of both approaches.