Modified LDA vector and feedback analysis for short query Information Retrieval systems

Abstract

Abstract Information Retrieval systems benefit from the use of long queries containing a large volume of search-relevant information. This situation is not common, as users of such systems tend to use very short and precise queries with few keywords. In this work we propose a modification of the Latent Dirichlet Allocation (LDA) technique using data from the document collection and its vocabulary for a better representation of short queries. Additionally, a study is carried out on how the modification of the proposed LDA weighted vectors increase the performance using relevant documents as feedback. The work shown in this paper is tested using three biomedical corpora (TREC Genomics 2004, TREC Genomics 2005 and OHSUMED) and one legal corpus (FIRE 2017). Results prove that the application of the proposed representation technique, as well as the feedback adjustment, clearly outperforms the baseline methods (BM25 and non-modified LDA).