Improving part-of-speech guessing of Chinese unknown words using hybrid models

Abstract

Listen

Translate article

This paper presents a hybrid model for part-of-speech (POS) guessing of Chinese unknown words. Most previous studies on this task have developed a unified statistical model for all Chinese unknown words and have rejected rule-based models without testing. We argue that models that use different sources of information about unknown words, both structural and contextual, can be effective for handling different types of unknown words. We propose a rule-based model that uses information about the type, length, and internal structure of unknown words and combine it with two existing statistical models that use information about the POS context and component characters of unknown words respectively for this task. By combining the complementary strengths of the three models that use different sources of information, the hybrid model achieves an accuracy of 89%, a significant improvement over the best result reported in previous studies.