Developing an automatic linguistic truncation operator for best-match retrieval of Finnish in inflected word form text database indexes

Abstract

The paper presents a new method for handling of morphological variation of query terms in best-match IR. The method is based on enhanced inflectional stems. Use of inflectional stems has earlier been shown to be a good retrieval method in inflected indexes in a best-match environment for a highly inflected and compound-rich language, Finnish. In this paper the earlier stem method is elaborated upon by enhancing the stems with regular expressions. Contrary to our expectations the results show that the enhanced stem queries do not outperform basic inflectional stems, but neither are they considerably worse with long queries. With short web-like queries they perform relatively better than with long queries and outperform clearly stemming (Finnish stemmer of Snowball) and plain, unprocessed query words. The main benefits of the proposed method, besides fairly good precision and recall (P-R) performance, are shorter and more manageable queries, which is of practical importance, e.g. with large web indexes.