Abstract
The balanced corpus of contemporary written Japanese (BCCWJ) is Japan's first 100 million words balanced corpus. It consists of three subcorpora (publication subcorpus, library subcorpus, and special-purpose subcorpus) and covers a wide range of text registers including books in general, magazines, newspapers, governmental white papers, best-selling books, an internet bulletin-board, a blog, school textbooks, minutes of the national diet, publicity newsletters of local governments, laws, and poetry verses. A random sampling technique is utilized whenever possible in order to maximize the representativeness of the corpus. The corpus is annotated in terms of dual POS analysis, document structure, and bibliographical information. The BCCWJ is currently accessible in three different ways including Chunagon a web-based interface to the dual POS analysis data. Lastly, results of some pilot evaluation of the corpus with respect to the textual diversity are reported. The analyses include POS distribution, word-class distribution, entropy of orthography, sentence length, and variation of the adjective predicate. High textual diversity is observed in all these analyses.
Highlights
One serious problem in the corpus-based analyses of present-day Japanese is the lack of a balanced corpus
The Kyoto University Text Corpus (Kurohashi and Nagao 1998) that played an important role in the development of an annotated corpus of Japanese natural language processing (NLP) consists of 40 thousands sentences taken from the articles of the Mainichi newspaper published in 1995
Whenever it is possible to divide a sample into separate parts that are written by different authors, each part is called an ‘article.’ Article information of the balanced corpus of contemporary written Japanese (BCCWJ) consists of fields like Article_ID, Directory_ID, First_appearance, First_published, and so forth
Summary
One serious problem in the corpus-based analyses of present-day Japanese is the lack of a balanced corpus. There is the possibility of estimating the meta-information by means of various up-to-date statistical clustering and classification methods, this approach requires a certain amount of supervised learning training data, derived from reliable reference corpora including the types of data mentioned above and covering various text types. To solve these problems, the authors launched a corpus compilation project in the spring of 2006, for public release of Japan’s first 100 million words balanced corpus in the year of 2011. The corpus was named the Balanced Corpus of Contemporary Written Japanese (BCCWJ, hereafter)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
