Research on the Method and System of Word Segmentation and POS Tagging for Ancient Chinese Medicine Literature

Abstract

Objectives: To better explore the valuable experience and knowledge in traditional Chinese medicine (TCM) literature and promote the academic heritage and innovation of TCM, this paper tries to provide methods and a system of word segmentation and POS tagging for ancient Chinese medicine literature. Method: Based on HMM and Java language for POS tagging, this paper develops a word segmentation system of Chinese ancient books by constructing a Thesaurus of Chinese medicine terminology and a special POS tagging method for Chinese medicine, and adopting Ansj open source code as the core word segmentation algorithm. Results: Thesaurus of Chinese medicine terminology involving 155,343 Chinese medicine words are constructed. These words were divided into 14 categories and 7 levels of 891 parts of speech (POS). Online website of Chinese word segmentation and POS tagging proprietary system were established (http://www.zhongyifenci.org). And the F measure reaches 90.65%, which is much higher than the system of Ansj. Clinical or Biological Impact: This system, with high precision and recall rate, will be very helpful to knowledge discovery of Chinese medicine and be beneficial to giving full play to the original advantages of Chinese medicine.