인공지능 기반 한문 번역을 위한 코퍼스 추출 및 정제 과정

Abstract

It is fun to imagine how good it would be to be able to translate a lot of Chinese classics by using AI in an era where the field of AI is expanding day by day. This is why commercialization of machine translation called artificial intelligence can drastically reduce the manpower and time spent on classical Chinese translation. To this end, various institutions are improving their performance by developing artificial intelligence Chinese character recognition and Chinese character translation programs. AI-based Chinese character translation, research is concentrated in the field of technology, and there are no reported studies on the corpus extraction process and purification process.
 For deep learning, which trains artificial intelligence, automatic translation data, that is, corpus that connects Chinese text and translations in parallel, must be created. In order to make a corpus, a lot of corpus data is extracted through human translation, and a high-quality corpus is made through refining based on detailed guidelines. Here, we tried to confirm the purification process of how the extracted corpus data were selected.
 As a result of checking the corpus purification process, words such as various titles, items, and names, metrological units or number of people were all excluded. Short sentences containing place names, human names, government posts, places, products, dates, and proper nouns were also removed because they were not appropriate as corpus. In addition, prefixes consisting of one or two letters, such as adverbs, conjunctions, tense, and pronounciation at the beginning of the sentence, were deleted. In the future, it is expected that the time and expense of translation will be greatly reduced if extraction is carried out in consideration of these contents in the construction of corpus for AI-based Chinese character translation.