A comprehensive review of content-based image retrieval systems using deep learning and hand-crafted features in medical imaging: Research challenges and future directions

Abstract

Computer-based medical image retrieval (CBMIR) system helps practitioners to enhance their diagnostic abilities, speeds up accurate diagnosis, and minimizes intra-and inter-observer variability. We investigate in-depth insights on content-based image retrieval systems over the past 20 years, to enable improvements with an imperative feature set to enhance the performance of the proposed CBMIR. After a comprehensive survey, we have made some inferences from the literature survey which is beneficial for researchers and medical practitioners. Existing studies, consider patch sizes of 32 only where 16 and 64 are ignored. We conduct experiments using 16, 32, and 64 patch sizes and two retrieval approaches i.e. simple distance-based retrieval (SDR) and customized query-based approach (CQA). SDR exhibits high inter-class ambiguity, in which some of the retrieved images are from different classes posing similar feature values, which negatively impacts the retrieval performance. To tackle this, a framework has been proposed based on CQA-CBMIR which utilizes wavelet-based Riesz features and improves the retrieval outcomes. The proposed framework is compared with the state-of-the-art (SOTA) methods and obtained f1-score and accuracy i.e. 86.5% and 86.8% respectively. Furthermore, this study illustrates some of the unresolved issues in the literature that demand future researchers' utmost attention to investigate new pathways in this discipline.