A novel text mining approach for scholar information extraction from web content in Chinese

Abstract

Text mining is the process of deriving high-quality information from text so that it can focus on extracting useful information from text or web documents. IoT devices generate massive structured or unstructured data including text data. The opportunity coming behind big data and unstructured data is a great impulse for governments or companies to choose solutions based on text mining approaches to improve strategic business activities and boost decision making. Expert information is an important reference information for decision making. How to collect the expert information from text or web documents is a problem. In this paper, a text mining approach is introduced to crawl and extract expert information from Internet. We build a basic framework and main modules including information extraction, data cleaning and deduplication, expert recommendation model to cope with text data from Web content. We also define several metrics, data structures and propose some algorithms to help text mining. Finally, the experiment is implemented with datasets and the results show that our text mining approach can extract expert attributes accurately.