Bridging the gap in biomedical information retrieval: Harnessing machine learning for enhanced search results and query semantics

Abstract

The medical domain faces unique challenges in Information Retrieval (IR) due to the complexity of medical language and terminology discrepancies between user queries and documents. While traditional Keyword-Based Methods (KBM) have limitations, the integration of semantic knowledge bases and concept mapping techniques enhances data organization and retrieval. Addressing the growing demands in the biomedical field, a novel medical Information Retrieval System (IRS) is proposed that employs Deep Learning (DL) and KBM. This system comprises five core steps: pre-processing of texts, document indexing using DL (ELMo) and KBM, advanced query processing, a BiLSTM-based retrieval network for contextual representation, and a KR-R re-ranking algorithm to refine document relevance. The purpose of the system is to give users improved biomedical search results through the integration of all of these techniques into a method that takes into consideration the semantic problems of medical records. An in-depth examination of the TREC-PM track samples from 2017 to 2019 observed an impressive leading MRR score of 0.605 in 2017 and a best-in-class rPrec score of 0.350 in 2019, proving how well able the system is to detect and rank relevant medical records accurately.