Recognition of english and russian-language texts based on frequency characteristics

Abstract

Distinction of texts in one language from texts in others is necessary to solve the problems of automated text analysis. The paper presents criteria and critical values for recognizing English-language and Russian-language texts. The obtained criteria are estimated by experiments. The paper describes the methods to estimate the size of character codes and to identify a space character in a text. The algorithm for recognizing texts in the English and Russian languages with arbitrary encoding is studied and its accuracy is estimated experimentally.