Estimation of the Closeness to a Semantic Pattern of a Topical Text without Construction of Periphrases

Abstract

The paper considers the problem of numerical estimation of the closeness of a topical text to the most rational linguistic variant (i.e. semantic pattern or sense standard) of the description of the knowledge fragment it represents without paraphrasing. This problem is relevant when implementing targeted selection of text information by the maximum of the useful semantic component with respect to the tasks solved by the user. Examples of practical applications may include selection of papers for scientific publishing and design of training courses and educational portals. In the suggested solution, the basis of the estimate of the closeness of the text to the semantic pattern is the splitting of the words of each of its phrases into classes by the TF-IDF metric value relative to texts of a corpus preformed by an expert. Abstracts of scientific papers together with their titles are analyzed. The suggested numerical estimate of closeness to the sense standard makes it possible to rank articles by the significance of the described fragments of knowledge regarding a given subject area and by non-redundancy of the description itself. Here, the semantic images of the texts closest to the semantic pattern specify the words with the highest TF-IDF values, which, when placed next to each other in the linear series of a phrase, are, most probably, semantically related and form key combinations with words whose mentioned metric is close to average. To classify word combinations as key ones, the interpretation of the TF-IDF metric, estimating the number of simultaneous occurrences of all words in the analyzed combination into phrases of the individual document, is introduced.