A method for the automatic extraction of keywords in legislative documents using statistical, semantic, and clustering relationships

Abstract

Using smart methods for the automatic generation of keywords in legislative documents has attracted the attention of many researchers over the past few decades. With the increasing evelopment of legislative documents and the large volume of unstructured texts, the need for rapid access to these documents has become more significant. Extracting the keywords in legislative documents will accel-erate the legislative process and reduce costs. Nowadays, many methods are presented dynamically for generating keywords. The present study attempted to extract more meaningful keywords from texts by using the thesaurus, which has a structured system to improve the classification of legislative documents. In this method, the semantic relationships in the thesaurus and document clustering were used and the statistical features of different words were calculated to extract some words as keywords. After pre-processing the texts, first the keywords in the text were selected using statisti-cal methods. Then, the phrases derived from the keywords were extracted using semantic terms in the thesaurus. After that, a numerical weight was assigned to each word to determine the relative importance of the words and indicate the effect of the word in relation to the text and compared to other words. Finally, the final keywords were selected using the relationships in the thesaurus and clustering methods. In order to evaluate this method, the tested text was compared to educational texts and the similarity between them was used. The results of testing various texts and subjects indicated the high accuracy of the proposed method. The data from the Parliament of Iran and the Deputy for Presidential Laws were used to evaluate the proposed model. This model could provide a very high accuracy and performance in these two bases in comparison to other methods.