Topic Modeling for Assessment of Text Complexity in Russian Textbooks

Abstract

This article explores the problem of assessing the complexity of Russian educational texts. We present a quantitative and qualitative investigation of topics in the textbooks of different grades by applying several topic models. The corpus compiled for the current study comprises a collection of textbooks used in middle and high schools of the Russian Federation. The corpus consists of two sets of texts derived from textbooks on Social science by L.N. Bogolyubov and A.F. Nikitin. For our analysis, we employ Latent Dirichlet Allocation (LDA) and Additive regularization of topic models (ARTM). In this paper, we use the grades of educational texts as indicators of text complexity. To identify quantitative measures of text complexity, we analyze the correlation of book grades with topic properties. We also analyze a correlation of a topic's properties with the growth of its proportion in a book of a specific complexity level. The key finding of our research is the identification of statistically significant correlations of educational texts complexity with the revealed topic features, i.e., topic coherence and distance between topic words in a semantic space. This study can be beneficial for two communities: textbook writers and their consumers, i.e., teachers and schoolchildren.