Mechanisms for optimization of detection and correction of text errors based on combining multilevel morphological analysis with n-gram models

Abstract

In the article the problem of increasing the information reliability in electronic document management systems is formulated, and mechanisms for controlling and correcting spelling and errors with semantic values are developed on the basis of a combined multilevel morphological analysis with n-gram models, a typical search, recognition, and classification tools. Mechanisms for verifying the spelling of a word on the basis of a vector representation of variables and comparison with a standard analogue are proposed according to the principles of using statistical, natural, structural, technological, semantic information redundancy. The solutions to the problems of increasing the information reliability based on a set of keywords, phrases, terms by comparing with virtual, frequency dictionaries located in the electronic document database and knowledge base are obtained. A technique has been developed to optimize control mechanisms and correct spelling errors based on the use of logical, semantic and structural - technological links, cross-relationships between individual or groups of words, phrases in the text information. The obtained tools to increase the reliability of the texts of electronic documents are tested in real condition, the results are compared with the conclusions of the system experts.