Abstract
The literature in the field of traditional Chinese medicine (TCM) contains a large amount of knowledge of traditional Chinese medicine. Such knowledge plays an important role in the automatic diagnosis and treatment of TCM. In order to obtain the above knowledge, the word segmentation of TCM texts is crucial and fundamental. However, the corpus of TCM texts presents mostly in the form of classical Chinese, which is different from modern Chinese, but there are still some cases where the path of modern Chinese and classical Chinese comes in cross. So it is very difficult to use the current universal word segmentation. The lack of manually labeled corpus for TCM word segmentation which is also hard to get makes it difficult to use a supervised method to train the tokenizer applied in the Chinese medicine. In order to solve this problem, we use an unsupervised method which uses entropy as goodness together with traditional Chinese medicine domain dictionary to construct a TCM text-specific tokenizer. Finally, the effectiveness of this method has been proved through experiments on 280 MB TCM texts.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
